Mar 26 00:58:01.643243 np0000160849 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
Mar 26 00:58:02.958583 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.common.config [-] Logging enabled!
Mar 26 00:58:02.959144 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev61
Mar 26 00:58:02.959144 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=104895) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
Mar 26 00:58:02.959144 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
Mar 26 00:58:02.959335 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=104895) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Mar 26 00:58:02.959335 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=104895) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Mar 26 00:58:02.959335 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=104895) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Mar 26 00:58:02.959492 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=104895) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Mar 26 00:58:02.959492 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=104895) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Mar 26 00:58:02.959621 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.manager [-] Loading core plugin: ml2
Mar 26 00:58:03.226457 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.226653 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.226653 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.227291 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.227360 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.227457 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.262706 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.262930 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.262930 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.269204 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.269287 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.269376 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.291289 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.291361 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.291414 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.408997 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.409106 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.409467 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.464585 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x76ba87586c00> agent after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.464585 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x76ba87586c00> agent after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.464585 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x76ba87586ca0> segment precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.464855 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x76ba87586d40> network precommit_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.464855 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x76ba87587f60> segment before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.514711 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> network before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.514813 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x76ba8776b420> agent before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.514877 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> network before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.514935 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> port before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515013 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> port before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515099 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> segment precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515099 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> segment precommit_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515226 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> segment after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515226 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> segment after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515423 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> network after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515423 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> network precommit_delete 0, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515423 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> port provisioning_complete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515583 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> rbac-policy before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515583 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> rbac-policy after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515722 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> agent after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515722 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> subnet after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515722 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> subnet precommit_delete 0, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515881 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_tenant of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> rbac-policy before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515881 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_tenant of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> rbac-policy before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.515881 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> segment_host_mapping after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.516051 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> agent after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.516051 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> rbac-policy before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.516306 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> rbac-policy before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.516306 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x76ba874944d0>> rbac-policy before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.516484 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=104895) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 26 00:58:03.516547 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=104895) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 26 00:58:03.516604 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=104895) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 26 00:58:03.516604 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=104895) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 26 00:58:03.516707 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=104895) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 26 00:58:03.516707 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=104895) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 26 00:58:03.516895 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
Mar 26 00:58:03.517728 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
Mar 26 00:58:03.518314 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
Mar 26 00:58:03.518846 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
Mar 26 00:58:03.518910 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
Mar 26 00:58:03.519073 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
Mar 26 00:58:03.519157 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
Mar 26 00:58:03.519267 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
Mar 26 00:58:03.519818 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
Mar 26 00:58:03.519879 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
Mar 26 00:58:03.520008 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
Mar 26 00:58:03.556369 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
Mar 26 00:58:03.556513 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
Mar 26 00:58:03.556513 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
Mar 26 00:58:03.556992 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x76ba872c5d00>> floatingip before_response 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.557082 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x76ba872c5d00>> port before_response 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:03.580489 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
Mar 26 00:58:03.580489 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
Mar 26 00:58:03.580695 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
Mar 26 00:58:03.580695 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
Mar 26 00:58:03.626326 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO dbcounter [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Registered counter for database neutron
Mar 26 00:58:03.642004 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG oslo_db.sqlalchemy.engines [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104895) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
Mar 26 00:58:04.035750 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG dbcounter [-] [104895] Writer thread running {{(pid=104895) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Mar 26 00:58:04.218153 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Initializing driver for type 'local'
Mar 26 00:58:04.218153 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Initializing driver for type 'vlan'
Mar 26 00:58:04.325766 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] VlanTypeDriver initialization complete
Mar 26 00:58:04.326009 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Initializing extension driver 'port_security'
Mar 26 00:58:04.326009 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.extensions.port_security [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] PortSecurityExtensionDriver initialization complete
Mar 26 00:58:04.326009 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Initializing extension driver 'qos'
Mar 26 00:58:04.326283 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.plugins.ml2.extensions.qos [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QosExtensionDriver initialization complete {{(pid=104895) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
Mar 26 00:58:04.326283 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Initializing mechanism driver 'ovn'
Mar 26 00:58:04.326283 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Starting OVNMechanismDriver
Mar 26 00:58:04.327703 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> process before_spawn 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.327829 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> process after_init 55550000, True {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.327907 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> segment after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328017 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> segment after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328144 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> segment after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328144 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> allowed_address_pair before_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328312 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> security_group precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328312 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> security_group after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328478 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> security_group after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328478 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> security_group precommit_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328665 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> security_group after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328665 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> security_group_rule precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328665 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> security_group_rule after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328945 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> security_group_rule before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328945 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> address_group after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328945 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> address_group after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.328945 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x76ba872c5f40>> address_group after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.329390 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x76ba8713e360>> qos_plugin after_init 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.329390 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x76ba86f76690>> trunk_plugin after_init 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.329565 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x76ba86548d40>> logging-plugin after_init 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.329942 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.services.logapi.drivers.ovn.driver [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] OVN logging driver registered
Mar 26 00:58:04.331448 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.plugins.ml2.plugin [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=104895) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Mar 26 00:58:04.331663 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548fb0>> port after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.331760 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548fb0>> port after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.331953 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548fb0>> port after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332064 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548f20>> subnet after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332126 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548f20>> subnet after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332184 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548f20>> subnet after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332243 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548ec0>> network after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332319 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548ec0>> network after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332412 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548ec0>> network after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332477 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548e90>> security_group after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332537 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548e90>> security_group after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332627 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548e90>> security_group after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332690 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548ef0>> security_group_rule after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332761 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548ef0>> security_group_rule after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332836 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548ef0>> security_group_rule after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.332938 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548e00>> address_group after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.333023 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548e00>> address_group after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.333099 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x76ba86548e00>> address_group after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.333168 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ML2 OVO RPC backend initialized. {{(pid=104895) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
Mar 26 00:58:04.333784 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.quota [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x76ba8654a690>.
Mar 26 00:58:04.333862 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.plugin [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Modular L2 Plugin initialization complete
Mar 26 00:58:04.333968 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading services supported by the core plugin {{(pid=104895) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
Mar 26 00:58:04.334062 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.plugins.ml2.managers [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Got port-security extension from driver 'port_security'
Mar 26 00:58:04.334287 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=104895) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
Mar 26 00:58:04.334354 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading Plugin: ovn-router
Mar 26 00:58:04.341350 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x76ba8654a720>> router before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.341455 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x76ba873df880> port after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.341510 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x76ba873df740> port after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.341591 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x76ba873df7e0> subnet after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.341681 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x76ba873df920> subnetpool_address_scope after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.341764 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x76ba8653a0c0> port before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.341824 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x76ba8653a0c0> port after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.341913 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x76ba8654a720>> port precommit_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.341993 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x76ba873c77e0> port before_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.342102 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x76ba873c7740> port before_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.342186 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x76ba8654a720>> router precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.342246 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x76ba8654a720>> router precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.342306 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x76ba8654a720>> router precommit_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.342394 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x76ba8653a020> subnet after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.343004 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.quota.resource_registry [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Tracking information for resource: router configured {{(pid=104895) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 26 00:58:04.343083 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.quota.resource_registry [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Tracking information for resource: floatingip configured {{(pid=104895) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Mar 26 00:58:04.343164 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.services.ovn_l3.plugin [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Starting OVNL3RouterPlugin
Mar 26 00:58:04.343529 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x76ba86517ef0>> port_forwarding_plugin after_init 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.343695 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x76ba8654a630>> router before_create 55550100, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.343763 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x76ba8654a630>> router precommit_delete 55550100, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.343863 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x76ba8654a630>> router precommit_create 55550100, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.343936 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x76ba8654a630>> router precommit_update 55550100, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.346550 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.services.provider_configuration [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Service providers = [] {{(pid=104895) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
Mar 26 00:58:04.346641 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.db.servicetype_db [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=104895) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}}
Mar 26 00:58:04.348219 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> router precommit_create 55549900, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.348343 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> router_interface after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.348343 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> floatingip after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.348640 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> floatingip precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.348702 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> floatingip precommit_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.348773 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> floatingip after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.348841 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> floatingip after_status_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.348899 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> floatingip after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.348957 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> port_forwarding after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.349063 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> router_interface after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.349125 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> router after_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.349341 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> router precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.349341 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> router after_delete 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.349341 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x76ba8654bc80>> router after_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.349519 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.services.service_base [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=104895) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
Mar 26 00:58:04.349519 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x76ba8654a720>> process after_init 55550001, True {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.349625 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=104895) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 26 00:58:04.349702 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading Plugin: auto_allocate
Mar 26 00:58:04.350444 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x76ba86558360> network precommit_update 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.350543 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron_lib.callbacks.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x76ba86558360> network precommit_create 55550000, False {{(pid=104895) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.350679 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=104895) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 26 00:58:04.350770 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading Plugin: tag
Mar 26 00:58:04.352882 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=104895) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 26 00:58:04.352948 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading Plugin: timestamp
Mar 26 00:58:04.353911 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=104895) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 26 00:58:04.353993 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading Plugin: network_ip_availability
Mar 26 00:58:04.355186 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=104895) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 26 00:58:04.355262 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading Plugin: flavors
Mar 26 00:58:04.355985 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=104895) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 26 00:58:04.356074 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading Plugin: revisions
Mar 26 00:58:04.356855 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.manager [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=104895) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Mar 26 00:58:04.356990 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG oslo_concurrency.lockutils [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.397s {{(pid=104895) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Mar 26 00:58:04.357143 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=104895) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
Mar 26 00:58:04.357218 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Initializing extension manager.
Mar 26 00:58:04.357438 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: __init__.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.357513 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: __pycache__ {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.357590 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: address_group.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.357864 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.358078 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: address-group
Mar 26 00:58:04.358140 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: address_scope.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.358388 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.358537 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: address-scope
Mar 26 00:58:04.358599 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: admin_state_down_before_update.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.358796 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.358964 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
Mar 26 00:58:04.359036 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: agent.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.359282 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.359420 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: agent
Mar 26 00:58:04.359490 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: agent_resources_synced.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.359671 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.359820 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension agent-resources-synced not supported by any of loaded plugins
Mar 26 00:58:04.359919 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: allowedaddresspairs.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.360620 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.360763 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: allowed-address-pairs
Mar 26 00:58:04.360829 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: auto_allocated_topology.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.361048 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.361180 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: auto-allocated-topology
Mar 26 00:58:04.361245 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: availability_zone.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.361479 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.361606 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: availability_zone
Mar 26 00:58:04.361666 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: availability_zone_filter.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.361862 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.361988 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension availability_zone_filter not supported by any of loaded plugins
Mar 26 00:58:04.362074 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: data_plane_status.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.362249 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.362380 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension data-plane-status not supported by any of loaded plugins
Mar 26 00:58:04.362439 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: default_subnetpools.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.362631 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.362755 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: default-subnetpools
Mar 26 00:58:04.362816 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: dhcpagentscheduler.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.363189 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.363408 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: dhcp_agent_scheduler
Mar 26 00:58:04.363408 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: dns.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.363558 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.363700 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: dns-integration
Mar 26 00:58:04.363764 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: dns_domain_ports.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.363940 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.364074 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: dns-domain-ports
Mar 26 00:58:04.364176 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.364326 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.364468 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: dns-integration-domain-keywords
Mar 26 00:58:04.364529 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: dvr.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.364730 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.364857 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension dvr not supported by any of loaded plugins
Mar 26 00:58:04.364916 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: ecmp_routes.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.365120 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.365251 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension ecmp_routes not supported by any of loaded plugins
Mar 26 00:58:04.365331 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: empty_string_filtering.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.365494 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.365626 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: empty-string-filtering
Mar 26 00:58:04.365684 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.365869 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.366013 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
Mar 26 00:58:04.366072 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.366257 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.366390 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
Mar 26 00:58:04.366450 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: external_net.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.366642 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.366764 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: external-net
Mar 26 00:58:04.366823 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: extra_dhcp_opt.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.367042 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.367179 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: extra_dhcp_opt
Mar 26 00:58:04.367238 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: extraroute.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.367423 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.367551 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: extraroute
Mar 26 00:58:04.367620 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: extraroute_atomic.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.367807 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.367934 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: extraroute-atomic
Mar 26 00:58:04.368020 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: filter_validation.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.368202 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.368333 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: filter-validation
Mar 26 00:58:04.368402 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: fip_pf_description.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.368576 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.368706 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
Mar 26 00:58:04.368776 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: fip_pf_detail.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.368953 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.369105 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: floating-ip-port-forwarding-detail
Mar 26 00:58:04.369174 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: fip_pf_port_range.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.369348 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.369479 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
Mar 26 00:58:04.369549 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: fip_port_details.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.369839 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.369990 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: fip-port-details
Mar 26 00:58:04.370100 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: flavors.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.370307 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.370443 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: flavors
Mar 26 00:58:04.370510 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.370763 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.370909 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
Mar 26 00:58:04.370988 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: floatingip_pools.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.371206 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.371341 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: floatingip-pools
Mar 26 00:58:04.371407 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: ip_allocation.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.371584 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.371713 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension ip_allocation not supported by any of loaded plugins
Mar 26 00:58:04.371825 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: ip_substring_port_filtering.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.371954 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.372104 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension ip-substring-filtering not supported by any of loaded plugins
Mar 26 00:58:04.372171 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l2_adjacency.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.372348 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.372482 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension l2_adjacency not supported by any of loaded plugins
Mar 26 00:58:04.372546 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.372869 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.373017 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: router
Mar 26 00:58:04.373095 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_conntrack_helper.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.373559 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.373699 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension l3-conntrack-helper not supported by any of loaded plugins
Mar 26 00:58:04.373771 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.373962 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.374121 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: enable-default-route-bfd
Mar 26 00:58:04.374185 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.374373 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.374504 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: enable-default-route-ecmp
Mar 26 00:58:04.374573 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_ext_gw_mode.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.374749 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.374921 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: ext-gw-mode
Mar 26 00:58:04.375012 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_ext_ha_mode.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.375191 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.375322 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: l3-ha
Mar 26 00:58:04.375392 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.375624 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.375751 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
Mar 26 00:58:04.375821 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_extra_gws.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.376014 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.376146 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: external-gateway-multihoming
Mar 26 00:58:04.376214 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_flavors.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.376388 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.376518 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: l3-flavors
Mar 26 00:58:04.376590 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_ndp_proxy.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.376812 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.376941 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension l3-ndp-proxy not supported by any of loaded plugins
Mar 26 00:58:04.377025 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.377201 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.377330 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
Mar 26 00:58:04.377400 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: l3agentscheduler.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.377823 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.377953 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension l3_agent_scheduler not supported by any of loaded plugins
Mar 26 00:58:04.378045 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: local_ip.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.378290 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.378425 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension local_ip not supported by any of loaded plugins
Mar 26 00:58:04.378490 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: logging.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.378728 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.378856 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension logging not supported by any of loaded plugins
Mar 26 00:58:04.378940 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: metering.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.379311 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.379447 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension metering not supported by any of loaded plugins
Mar 26 00:58:04.379501 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.379705 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.379840 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
Mar 26 00:58:04.379904 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: multiprovidernet.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.380107 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.380248 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: multi-provider
Mar 26 00:58:04.380313 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: netmtu.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.380488 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.380620 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: net-mtu
Mar 26 00:58:04.380689 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: netmtu_writable.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.380862 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.381013 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: net-mtu-writable
Mar 26 00:58:04.381086 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: network_availability_zone.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.381304 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.381432 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: network_availability_zone
Mar 26 00:58:04.381502 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: network_ha.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.381675 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.381806 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension network_ha not supported by any of loaded plugins
Mar 26 00:58:04.381877 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: network_ip_availability.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.382087 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.382223 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: network-ip-availability
Mar 26 00:58:04.382293 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: network_ip_availability_details.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.382484 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.382616 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: network-ip-availability-details
Mar 26 00:58:04.382687 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: network_segment_range.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.382962 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.383114 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension network-segment-range not supported by any of loaded plugins
Mar 26 00:58:04.383193 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: pagination.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.383427 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.383529 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: pagination
Mar 26 00:58:04.383623 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_device_profile.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.383856 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.384061 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: port-device-profile
Mar 26 00:58:04.384160 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_hardware_offload_type.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.384394 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.384567 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension port-hardware-offload-type not supported by any of loaded plugins
Mar 26 00:58:04.384662 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.384886 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.385090 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
Mar 26 00:58:04.385185 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_hints.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.385408 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.385583 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension port-hints not supported by any of loaded plugins
Mar 26 00:58:04.385678 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_mac_address_override.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.385901 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.386099 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension port-mac-override not supported by any of loaded plugins
Mar 26 00:58:04.386194 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_mac_address_regenerate.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.386419 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.386597 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: port-mac-address-regenerate
Mar 26 00:58:04.386696 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_numa_affinity_policy.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.386959 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.387166 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: port-numa-affinity-policy
Mar 26 00:58:04.387260 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.387487 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.387661 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: port-numa-affinity-policy-socket
Mar 26 00:58:04.387757 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_resource_request.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.387995 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.388171 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension port-resource-request not supported by any of loaded plugins
Mar 26 00:58:04.388264 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_resource_request_groups.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.388503 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.388680 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension port-resource-request-groups not supported by any of loaded plugins
Mar 26 00:58:04.388776 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: port_trusted_vif.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.389022 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.389199 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension port-trusted-vif not supported by any of loaded plugins
Mar 26 00:58:04.389295 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: portbindings.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.389521 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.389691 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: binding
Mar 26 00:58:04.389783 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: portbindings_extended.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.390065 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.390233 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: binding-extended
Mar 26 00:58:04.390333 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: portsecurity.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.390539 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.390678 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: port-security
Mar 26 00:58:04.390743 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: project_id.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.391040 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.391040 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: project-id
Mar 26 00:58:04.391173 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: providernet.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.391298 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.391434 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: provider
Mar 26 00:58:04.391500 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qinq.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.391697 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.391831 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: qinq
Mar 26 00:58:04.391898 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.392667 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.392799 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos not supported by any of loaded plugins
Mar 26 00:58:04.392870 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_bw_limit_direction.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.393105 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.393242 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
Mar 26 00:58:04.393308 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.393507 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.393638 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
Mar 26 00:58:04.393703 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_default.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.393897 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.394062 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-default not supported by any of loaded plugins
Mar 26 00:58:04.394136 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_fip.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.394322 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.394459 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-fip not supported by any of loaded plugins
Mar 26 00:58:04.394525 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_fip_network_policy.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.394701 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.394839 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-fip-network-policy not supported by any of loaded plugins
Mar 26 00:58:04.394917 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_gateway_ip.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.395138 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.395271 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-gateway-ip not supported by any of loaded plugins
Mar 26 00:58:04.395341 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_port_network_policy.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.395528 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.395663 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-port-network-policy not supported by any of loaded plugins
Mar 26 00:58:04.395730 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.395923 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.396094 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-pps-minimum not supported by any of loaded plugins
Mar 26 00:58:04.396166 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.396359 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.396492 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
Mar 26 00:58:04.396563 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_pps_rule.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.396757 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.396894 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-pps not supported by any of loaded plugins
Mar 26 00:58:04.396966 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_rule_type_details.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.397171 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.397304 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-rule-type-details not supported by any of loaded plugins
Mar 26 00:58:04.397385 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_rule_type_filter.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.397595 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.397762 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-rule-type-filter not supported by any of loaded plugins
Mar 26 00:58:04.397860 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: qos_rules_alias.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.398121 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.398286 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension qos-rules-alias not supported by any of loaded plugins
Mar 26 00:58:04.398374 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: quota_check_limit.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.398581 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.398712 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: quota-check-limit
Mar 26 00:58:04.398784 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: quota_check_limit_default.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.399075 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.399219 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: quota-check-limit-default
Mar 26 00:58:04.399290 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: quotasv2.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.399862 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.399942 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: quotas
Mar 26 00:58:04.400044 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: quotasv2_detail.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.400316 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.400393 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: quota_details
Mar 26 00:58:04.400460 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: rbac.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.400862 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control tenant access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.401023 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: rbac-policies
Mar 26 00:58:04.401103 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: rbac_address_group.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.401297 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.401439 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension rbac-address-group not supported by any of loaded plugins
Mar 26 00:58:04.401507 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: rbac_address_scope.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.401695 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.401826 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: rbac-address-scope
Mar 26 00:58:04.401895 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: rbac_security_groups.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.402116 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.402248 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: rbac-security-groups
Mar 26 00:58:04.402320 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: rbac_subnetpool.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.402500 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.402635 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension rbac-subnetpool not supported by any of loaded plugins
Mar 26 00:58:04.402709 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: revisionifmatch.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.402916 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.403077 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: revision-if-match
Mar 26 00:58:04.403158 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: revisions.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.403358 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.403529 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: standard-attr-revisions
Mar 26 00:58:04.403604 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: router_availability_zone.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.403814 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.403953 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: router_availability_zone
Mar 26 00:58:04.404075 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: router_enable_snat.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.404267 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.404406 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: router-enable-snat
Mar 26 00:58:04.404546 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: routerservicetype.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.404651 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.404785 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension router-service-type not supported by any of loaded plugins
Mar 26 00:58:04.404857 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: security_groups_default_rules.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.405249 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.405388 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: security-groups-default-rules
Mar 26 00:58:04.405451 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.405643 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.405781 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: security-groups-normalized-cidr
Mar 26 00:58:04.405863 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: security_groups_port_filtering.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.406089 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.406223 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension port-security-groups-filtering not supported by any of loaded plugins
Mar 26 00:58:04.406292 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: security_groups_remote_address_group.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.406485 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.406632 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: security-groups-remote-address-group
Mar 26 00:58:04.406704 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.406907 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.407079 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: security-groups-rules-belongs-to-default-sg
Mar 26 00:58:04.407168 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: security_groups_shared_filtering.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.407356 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.407492 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: security-groups-shared-filtering
Mar 26 00:58:04.407561 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: securitygroup.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.409610 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.409610 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: security-group
Mar 26 00:58:04.409610 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: segment.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.409610 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.409610 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension segment not supported by any of loaded plugins
Mar 26 00:58:04.409786 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.409862 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.410021 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
Mar 26 00:58:04.410101 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: servicetype.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.410308 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.410439 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: service-type
Mar 26 00:58:04.410514 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: sorting.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.410697 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.410769 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: sorting
Mar 26 00:58:04.410837 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: standard_attr_segment.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.411089 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.411226 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension standard-attr-segment not supported by any of loaded plugins
Mar 26 00:58:04.411296 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: standardattrdescription.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.411495 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.411635 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: standard-attr-description
Mar 26 00:58:04.411693 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: stateful_security_group.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.411877 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.412044 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: stateful-security-group
Mar 26 00:58:04.412126 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: stdattrs_common.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.412311 np0000160849 neutron-ovn-maintenance-worker[104895]: WARNING neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
Mar 26 00:58:04.412386 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.412566 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.412704 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: subnet-dns-publish-fixed-ip
Mar 26 00:58:04.412773 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: subnet_external_network.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.412947 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.413102 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: subnet-external-network
Mar 26 00:58:04.413171 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: subnet_onboard.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.413501 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.413640 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension subnet_onboard not supported by any of loaded plugins
Mar 26 00:58:04.413715 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: subnet_segmentid_writable.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.413895 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.414049 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
Mar 26 00:58:04.414118 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: subnet_service_types.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.414304 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.414436 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: subnet-service-types
Mar 26 00:58:04.414508 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: subnetallocation.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.414698 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.414838 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: subnet_allocation
Mar 26 00:58:04.414927 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.415152 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.415288 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: subnetpool-prefix-ops
Mar 26 00:58:04.415358 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: tag_creation.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.415533 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.415667 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: tag-creation
Mar 26 00:58:04.415737 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.415913 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.416068 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
Mar 26 00:58:04.416141 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: tagging.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.416787 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.416926 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: standard-attr-tag
Mar 26 00:58:04.417009 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: timestamp.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.417243 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.417381 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: standard-attr-timestamp
Mar 26 00:58:04.417445 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: trunk.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.417651 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.417773 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension trunk not supported by any of loaded plugins
Mar 26 00:58:04.417844 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: trunk_details.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.418042 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.418179 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension trunk-details not supported by any of loaded plugins
Mar 26 00:58:04.418246 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: uplink_status_propagation.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.418424 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.418558 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension uplink-status-propagation not supported by any of loaded plugins
Mar 26 00:58:04.418628 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.418805 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.418964 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
Mar 26 00:58:04.419054 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loading extension file: vlantransparent.py {{(pid=104895) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Mar 26 00:58:04.419259 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=104895) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Mar 26 00:58:04.419388 np0000160849 neutron-ovn-maintenance-worker[104895]: INFO neutron.api.extensions [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Loaded extension: vlan-transparent
Mar 26 00:58:04.421718 np0000160849 neutron-ovn-maintenance-worker[104895]: WARNING oslo_service.backend._threading.service [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
Mar 26 00:58:04.422100 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Full set of CONF: {{(pid=104895) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
Mar 26 00:58:04.422202 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ******************************************************************************** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Mar 26 00:58:04.422295 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Configuration options gathered from: {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Mar 26 00:58:04.422397 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Mar 26 00:58:04.422535 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Mar 26 00:58:04.422623 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ================================================================================ {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Mar 26 00:58:04.422777 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] agent_down_time                = 75 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.422963 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] allow_automatic_dhcp_failover  = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.423114 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] allow_bulk                     = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.423345 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.423473 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] api_extensions_path            =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.423602 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] api_paste_config               = api-paste.ini {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.423731 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] api_workers                    = 4 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.423853 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] auth_strategy                  = keystone {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.423967 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] backlog                        = 4096 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.424122 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.424262 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] bind_host                      = 0.0.0.0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.424382 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] bind_port                      = 9696 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.424498 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] client_socket_timeout          = 900 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.424624 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] config_dir                     = [] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.424715 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.424835 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] config_source                  = [] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.424968 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] control_exchange               = neutron {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.425116 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] core_plugin                    = ml2 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.425242 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] debug                          = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.425355 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] default_availability_zones     = [] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.425549 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.425665 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] dhcp_agent_notification        = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.425780 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] dhcp_agents_per_network        = 1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.425895 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] dhcp_lease_duration            = 86400 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426041 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] dhcp_load_type                 = networks {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426161 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] dns_domain                     = openstacklocal {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426286 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426396 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] enable_default_route_bfd       = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426505 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] enable_default_route_ecmp      = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426616 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] enable_dvr                     = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426726 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] enable_new_agents              = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426871 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] enable_services_on_agents_with_admin_state_down = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.426971 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] enable_snat_by_default         = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.427111 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] enable_traditional_dhcp        = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.427221 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] external_dns_driver            = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.427331 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] filter_validation              = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.427454 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] global_physnet_mtu             = 1430 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.427537 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] graceful_shutdown_timeout      = 60 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.427707 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] host                           = np0000160849 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.427824 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] host_dvr_for_dhcp              = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.427937 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] http_retries                   = 3 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.428078 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] instance_format                = [instance: %(uuid)s]  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.428211 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.428332 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ipam_driver                    = internal {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.428450 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ipv6_pd_enabled                = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.428578 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] l3_ha                          = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.428697 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.428815 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] l3_ha_network_physical_name    =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.428931 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] l3_ha_network_type             =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429075 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_color                      = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429175 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_config_append              = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429262 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429347 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_dir                        = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429431 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_file                       = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429494 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_options                    = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429582 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_rotate_interval            = 1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429673 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_rotate_interval_type       = days {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429768 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] log_rotation_type              = none {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429821 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429863 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.429963 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430092 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430161 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430237 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] max_allowed_address_pair       = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430321 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] max_dns_nameservers            = 5 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430404 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] max_header_line                = 16384 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430489 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] max_l3_agents_per_router       = 3 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430573 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] max_logfile_count              = 30 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430657 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] max_logfile_size_mb            = 200 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430752 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] max_routes                     = 30 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430859 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] max_subnet_host_routes         = 20 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.430914 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] my_ip                          = 199.204.45.35 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431057 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:febf:969f {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431138 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] network_auto_schedule          = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431224 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] network_link_prefix            = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431311 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431398 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] notify_nova_on_port_data_changes = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431485 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] notify_nova_on_port_status_changes = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431571 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] pagination_max_limit           = -1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431653 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] periodic_fuzzy_delay           = 5 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431736 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] periodic_interval              = 40 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431819 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] publish_errors                 = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.431910 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] rate_limit_burst               = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432005 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] rate_limit_except_level        = CRITICAL {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432109 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] rate_limit_interval            = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432191 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] retry_until_window             = 30 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432275 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] router_distributed             = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432358 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] rpc_resources_processing_step  = 20 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432441 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] rpc_response_max_timeout       = 600 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432525 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] rpc_response_timeout           = 60 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432615 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] rpc_state_report_workers       = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432697 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] rpc_workers                    = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432779 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] send_events_interval           = 2 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432872 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] service_plugins                = ['ovn-router'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.432961 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] setproctitle                   = on {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433061 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] shell_completion               = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433164 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] state_path                     = /opt/stack/data/neutron {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433247 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] syslog_log_facility            = LOG_USER {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433331 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] tcp_keepidle                   = 600 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433420 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] transport_url                  = **** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433503 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] use_journal                    = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433587 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] use_json                       = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433671 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] use_ssl                        = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433755 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] use_stderr                     = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433844 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] use_syslog                     = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.433926 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] vlan_qinq                      = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.434277 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] vlan_transparent               = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.434277 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] wsgi_default_pool_size         = 100 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.434277 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] wsgi_keep_alive                = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.434404 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.434404 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] wsgi_server_debug              = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.434507 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_concurrency.disable_process_locking = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.434612 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.434717 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.connection_string     = messaging:// {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.434810 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.enabled               = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.434946 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.es_doc_type           = notification {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435069 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.es_scroll_size        = 10000 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435163 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.es_scroll_time        = 2m {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435251 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.filter_error_trace    = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435341 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.hmac_keys             = **** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435426 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.sentinel_service_name = mymaster {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435522 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.socket_timeout        = 0.1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435609 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.trace_requests        = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435693 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler.trace_sqlalchemy      = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435791 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler_jaeger.process_tags   = {} {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435876 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler_jaeger.service_name_prefix = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.435964 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] profiler_otlp.service_name_prefix = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.436088 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.436176 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.436269 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.436361 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.436449 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.436547 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.enforce_new_defaults = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.436733 np0000160849 neutron-ovn-maintenance-worker[104895]: WARNING oslo_config.cfg [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
Mar 26 00:58:04.436828 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.enforce_scope      = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.436919 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.policy_default_rule = default {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437048 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437143 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437235 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437319 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437403 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437487 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437572 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437662 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_policy.remote_timeout     = 60.0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437758 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437850 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] service_providers.service_provider = [] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.437946 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.auth_section              = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438073 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.auth_type                 = password {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438167 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.auth_url                  = https://199.204.45.35/identity {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438279 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.cafile                    = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438344 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.certfile                  = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438427 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.collect_timing            = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438512 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.default_domain_id         = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438598 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.default_domain_name       = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438685 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.domain_id                 = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438765 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.domain_name               = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438882 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.endpoint_type             = public {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.438970 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.insecure                  = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439098 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.keyfile                   = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439199 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.password                  = **** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439286 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.project_domain_id         = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439375 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.project_domain_name       = Default {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439464 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.project_id                = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439557 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.project_name              = service {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439646 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.region_name               = RegionOne {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439729 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.split_loggers             = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439813 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.system_scope              = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439897 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.timeout                   = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.439982 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.trust_id                  = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.440097 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.user_domain_id            = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.440206 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.user_domain_name          = Default {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.440319 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.user_id                   = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.440472 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] nova.username                  = nova {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.440560 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.auth_section         = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.440684 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.auth_type            = password {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.440799 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.cafile               = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.440904 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.certfile             = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441035 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.collect_timing       = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441152 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.endpoint_type        = public {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441256 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.insecure             = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441361 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.keyfile              = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441465 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.region_name          = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441581 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.split_loggers        = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441693 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] placement.timeout              = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441817 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.auth_section            = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.441941 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.auth_type               = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.442083 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.cafile                  = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.442203 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.certfile                = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.442326 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.collect_timing          = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.442446 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.connect_retries         = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.442567 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.connect_retry_delay     = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.442690 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.enable_notifications    = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.442814 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.endpoint_override       = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.442958 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.insecure                = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.443109 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.interface               = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.443220 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.keyfile                 = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.443335 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.max_version             = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.443447 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.min_version             = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.443557 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.region_name             = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.443668 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.retriable_status_codes  = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.443779 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.service_name            = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.443891 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.service_type            = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444025 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.split_loggers           = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444144 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.status_code_retries     = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444263 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.status_code_retry_delay = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444376 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.timeout                 = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444494 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.valid_interfaces        = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444610 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ironic.version                 = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444733 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] cli_script.dry_run             = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444860 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.444995 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.445121 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.445235 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.445347 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.445467 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.445587 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.445703 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.445823 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.445949 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.hostname = np0000160849 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.446088 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.446213 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.446335 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.446460 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.446801 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.446801 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.446888 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.447009 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.447149 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.447273 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.447396 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.447517 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.447633 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.447749 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.447866 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.448001 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.448123 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.448448 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.448552 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.448670 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.448797 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.ssl      = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.448920 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.449060 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.449191 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.449309 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.449429 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.ssl_version =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.449547 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.449676 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_notifications.driver = [] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.449792 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_notifications.retry = -1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.449926 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.450294 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_messaging_notifications.transport_url = **** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.450294 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_reports.file_event_handler = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.450294 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_reports.file_event_handler_interval = 1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.450443 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_reports.log_dir           = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.450542 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.450654 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.default_quota           = -1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.450778 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.450930 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_floatingip        = 50 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.451072 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_network           = 100 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.451189 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_port              = 500 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.451492 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.451605 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_router            = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.451720 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_security_group    = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.451832 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_security_group_rule = 100 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.451948 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.quota_subnet            = 100 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.452082 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] QUOTAS.track_quota_usage       = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.452220 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.452338 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep.group                  = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.452461 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep.helper_command         = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.452585 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep.log_daemon_traceback   = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.452715 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.452840 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep.thread_pool_size       = 16 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.452963 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep.user                   = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.453118 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.453242 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_dhcp_release.group     = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.453364 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_dhcp_release.helper_command = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.453491 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.453620 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.453740 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.453864 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_dhcp_release.user      = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.454012 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.454136 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_ovs_vsctl.group        = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.454256 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_ovs_vsctl.helper_command = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.454384 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.454517 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.454650 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.454780 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_ovs_vsctl.user         = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.454956 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_namespace.capabilities = [21] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.455102 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_namespace.group        = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.455227 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_namespace.helper_command = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.455356 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_namespace.log_daemon_traceback = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.455495 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.455624 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_namespace.thread_pool_size = 16 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.455746 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_namespace.user         = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.455886 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_conntrack.capabilities = [12] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.456037 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_conntrack.group        = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.456162 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_conntrack.helper_command = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.456283 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_conntrack.log_daemon_traceback = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.456417 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.456544 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_conntrack.thread_pool_size = 16 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.456666 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_conntrack.user         = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.456807 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_link.capabilities      = [12, 21] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.456938 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_link.group             = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.457093 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_link.helper_command    = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.457223 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_link.log_daemon_traceback = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.457340 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.457458 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_link.thread_pool_size  = 16 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.457586 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] privsep_link.user              = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.457732 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.457869 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.458024 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] AGENT.use_helper_for_ns_read   = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.458162 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] SECURITYGROUP.enable_ipset     = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.458488 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] SECURITYGROUP.enable_security_group = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.458596 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] SECURITYGROUP.firewall_driver  = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.458733 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.459241 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] experimental.ipv6_pd_enabled   = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.459653 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.459878 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.external_network_type      = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.460137 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.460658 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.overlay_ip_version         = 4 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.460940 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.path_mtu                   = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.461197 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.physical_network_mtus      = {} {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.461418 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.project_network_types      = ['geneve'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.461619 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.461826 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.462063 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.462415 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2_type_geneve.max_header_size = 38 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.462611 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.462810 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.463127 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] network_log.burst_limit        = 25 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.463314 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] network_log.local_output_log_base = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.463495 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] network_log.rate_limit         = 100 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.463689 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.asyncio_connection    = **** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.463866 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.asyncio_slave_connection = **** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.464096 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.backend               = sqlalchemy {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.464287 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.connection            = **** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.464472 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.connection_debug      = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.464660 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.connection_parameters =  {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.464842 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.connection_recycle_time = 3600 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.465050 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.connection_trace      = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.465238 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.db_inc_retry_interval = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.465419 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.db_max_retries        = 20 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.465599 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.db_max_retry_interval = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.465778 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.db_retry_interval     = 1 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.465957 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.max_overflow          = 50 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.466168 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.max_pool_size         = 5 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.466350 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.max_retries           = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.466537 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.466714 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.mysql_wsrep_sync_wait = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.466970 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.pool_timeout          = None {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.467263 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.retry_interval        = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.467458 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.slave_connection      = **** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.467645 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.sqlite_synchronous    = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.467835 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.synchronous_reader    = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.468056 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] database.use_db_reconnect      = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.468343 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.bfd_min_rx                 = 1000 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.468581 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.bfd_min_tx                 = 100 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.468770 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.bfd_mult                   = 3 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.468958 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.broadcast_arps_to_all_routers = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.469185 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.469376 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.469599 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.dns_records_ovn_owned      = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.469840 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.dns_servers                = [] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.470072 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.enable_distributed_floating_ip = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.470265 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.fdb_age_threshold          = 300 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.470468 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ha_failover_strategy       = normal {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.470667 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.live_migration_activation_strategy = rarp {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.470929 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.localnet_learn_fdb         = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.471198 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.mac_binding_age_threshold  = 300 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.471405 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.neutron_sync_mode          = log {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.471603 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.471794 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.472018 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.472320 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_metadata_enabled       = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.472565 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.472861 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.473135 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_nb_connection          = ['ssl:199.204.45.35:6641'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.473367 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.473697 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_router_indirect_snat   = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.474033 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.474349 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.474658 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_sb_connection          = ['ssl:199.204.45.35:6642'] {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475072 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475216 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovs_create_tap             = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475340 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475472 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovsdb_log_level            = INFO {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475571 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475655 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475742 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.stateless_nat_enabled      = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475834 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.475929 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] OVS.bridge_mac_table_size      = 50000 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476044 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] OVS.igmp_flood                 = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476153 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] OVS.igmp_flood_reports         = True {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476241 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] OVS.igmp_flood_unregistered    = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476335 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] OVS.igmp_snooping_enable       = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476423 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] OVS.ovsdb_timeout              = 10 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476516 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476605 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476695 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.476761 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG cotyledon.oslo_config_glue [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] ******************************************************************************** {{(pid=104895) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Mar 26 00:58:04.476857 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG oslo_service.backend._threading.service [None req-6e1b6038-c049-4c47-861e-ef299301276b None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x76ba8b9ec830> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x76ba8b9ec830> {{(pid=104895) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
Mar 26 00:58:04.502757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=105681) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
Mar 26 00:58:04.503444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Mar 26 00:58:04.503444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Mar 26 00:58:04.503444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Mar 26 00:58:04.503444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Mar 26 00:58:04.503444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Mar 26 00:58:04.503834 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.503834 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.503834 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.504027 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.504098 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.504201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.504361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.504489 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.504580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] backlog                        = 4096 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.504696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.504893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505263 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505335 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505566 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505566 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.505875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.506122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.506366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.506366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.506366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.506696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.506696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.506696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507426 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.507730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000160849 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.508756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509294 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.509793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510570 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.510918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 199.204.45.35 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:febf:969f {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.511912 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512043 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512430 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512724 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] retry_until_window             = 30 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.512941 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513070 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513378 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513489 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.513903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514277 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514454 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] use_ssl                        = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.514909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.515109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.515109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.515228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.515297 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Mar 26 00:58:04.515472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.515645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.515785 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.515908 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516138 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516343 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516547 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.516874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517414 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517625 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517722 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.517914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518075 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518276 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518403 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518887 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.518953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://199.204.45.35/identity {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519043 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519231 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519231 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519445 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519445 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519552 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519552 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519863 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.519971 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520107 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520231 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.520935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521198 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521304 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521531 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521637 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521741 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.521963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522048 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522276 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522397 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522681 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522735 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.522929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523075 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523075 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523075 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000160849 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523392 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523465 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523760 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523840 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523912 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.523948 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524040 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.524930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525209 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525330 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525418 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525464 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525508 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525725 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525815 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525815 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.525960 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526089 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526125 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526322 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526357 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526464 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526536 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526577 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526655 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526902 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.526945 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527041 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527107 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527193 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527460 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527505 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527902 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527941 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.527997 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528107 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528153 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528301 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528339 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528417 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528519 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528556 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528593 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528629 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528770 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.528994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529154 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529356 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529584 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529848 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529884 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529919 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.529961 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530077 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530156 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:199.204.45.35:6641'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530271 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530342 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:199.204.45.35:6642'] {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530433 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530617 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530653 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530883 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.530958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.531056 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.531127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.531177 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.531231 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.531276 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Mar 26 00:58:04.531318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105681) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Mar 26 00:58:04.531493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [105681] {{(pid=105681) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
Mar 26 00:58:04.532781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-235486', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-29512'] for process (None), after_init {{(pid=105681) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Mar 26 00:58:04.534990 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
Mar 26 00:58:04.580183 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 26 00:58:04.580293 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 26 00:58:04.580351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 26 00:58:04.580448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.580545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.580639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.580719 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.580793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.580885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.580973 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.581064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.581149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.581220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.581274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.581365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.585130 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.35:6641: connecting...
Mar 26 00:58:04.590406 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.35:6641: connected
Mar 26 00:58:04.617467 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
Mar 26 00:58:04.648468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 26 00:58:04.648468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Mar 26 00:58:04.648680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.648680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=105681) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Mar 26 00:58:04.650879 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.35:6642: connecting...
Mar 26 00:58:04.655514 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.35:6642: connected
Mar 26 00:58:04.697602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x76ba85d2b2f0>> floatingip precommit_update 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.698005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x76ba85d2b2f0>> floatingip precommit_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.698136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x76ba85d2b2f0>> floatingip before_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.698223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x76ba85d2b2f0>> floatingip before_update 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.698312 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x76ba85d2b2f0>> port after_update 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.698444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x76ba85d2b2f0>> port precommit_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.698775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-12077'] for port_forwarding_plugin (None), after_init {{(pid=105681) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Mar 26 00:58:04.698887 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x76ba86517ef0>> port_forwarding after_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.698990 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x76ba86517ef0>> router_interface after_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.699112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x76ba86517ef0>> port_forwarding after_update 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.699197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x76ba86517ef0>> port_forwarding after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.699282 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x76ba86517ef0>> router_interface after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.699474 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Mar 26 00:58:04.699593 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
Mar 26 00:58:04.700153 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x76ba85d2b260>> segment before_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.700258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x76ba85d2b260>> subnet precommit_delete_associations 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.700665 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> segment_host_mapping after_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.700780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> segment_host_mapping after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.700903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> port after_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.701019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> port after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.701120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> port after_update 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.701205 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> segment after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.701299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> subnet after_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.701383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> subnet after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.701487 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x76ba86a1a180>> subnet after_update 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.765611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x76ba85d24350>> subnet after_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.765780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x76ba85d24350>> subnet after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.765780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x76ba85d24350>> subnet before_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.765780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x76ba85d24350>> subnet before_update 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.768029 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x76ba85d2a0f0>> security_group after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.768149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x76ba85d2a0f0>> port after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.768226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-1314010'] for logging-plugin (None), after_init {{(pid=105681) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Mar 26 00:58:04.768297 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x76ba85d29010>> security_group_rule after_create 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.768424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x76ba85d29010>> security_group_rule after_update 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:04.768424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x76ba85d29010>> security_group_rule after_delete 55550000, False {{(pid=105681) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Mar 26 00:58:14.222717 np0000160849 neutron-ovn-maintenance-worker[104895]: DEBUG dbcounter [-] [104895] Writing DB stats neutron:SELECT=4 {{(pid=104895) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 00:58:14.769548 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-03-26 00:58:14.769172 {{(pid=105681) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
Mar 26 00:58:14.772196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync port groups started @ 2026-03-26 00:58:14.771785 {{(pid=105681) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
Mar 26 00:58:14.790233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writer thread running {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Mar 26 00:58:14.806177 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync port groups completed @ 2026-03-26 00:58:14.805898 {{(pid=105681) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
Mar 26 00:58:14.806322 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-03-26 00:58:14.806151 {{(pid=105681) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1125}}
Mar 26 00:58:14.912727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.services.segments.db [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x76ba85d24ad0>,) {'filters': {'network_id': ['7c327d4f-b960-49ee-8c2b-fc5ec1b488ad'], 'is_dynamic': False}} {{(pid=105681) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Mar 26 00:58:14.917314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync metadata ports started {{(pid=105681) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1070}}
Mar 26 00:58:14.979676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync metadata ports completed {{(pid=105681) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1122}}
Mar 26 00:58:14.979818 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=105681) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:922}}
Mar 26 00:58:15.216419 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-03-26 00:58:15.216100 {{(pid=105681) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:995}}
Mar 26 00:58:15.217233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Mar 26 00:58:15.217591 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=105681) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1000}}
Mar 26 00:58:15.217774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=105681) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1059}}
Mar 26 00:58:15.217928 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-03-26 00:58:15.217764 {{(pid=105681) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1294}}
Mar 26 00:58:15.218147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync ACLs started @ 2026-03-26 00:58:15.217952 {{(pid=105681) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
Mar 26 00:58:15.252428 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync ACLs completed @ 2026-03-26 00:58:15.252181 {{(pid=105681) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:429}}
Mar 26 00:58:15.252572 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync routers and router ports started @ 2026-03-26 00:58:15.252422 {{(pid=105681) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:596}}
Mar 26 00:58:15.303402 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Mar 26 00:58:15.303679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync routers and router ports completed @ 2026-03-26 00:58:15.303449 {{(pid=105681) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:917}}
Mar 26 00:58:15.303814 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync port QoS policies started @ 2026-03-26 00:58:15.303654 {{(pid=105681) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1342}}
Mar 26 00:58:15.303897 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] Starting OVNClientQosExtension
Mar 26 00:58:15.321865 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync port QoS policies completed @ 2026-03-26 00:58:15.321650 {{(pid=105681) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1365}}
Mar 26 00:58:15.321964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync Floating IP QoS policies started @ 2026-03-26 00:58:15.321849 {{(pid=105681) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1370}}
Mar 26 00:58:15.322109 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] Starting OVNClientQosExtension
Mar 26 00:58:15.344667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-03-26 00:58:15.344466 {{(pid=105681) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1393}}
Mar 26 00:58:15.344667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync Floating IP NAT rules started @ 2026-03-26 00:58:15.344613 {{(pid=105681) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1398}}
Mar 26 00:58:15.344959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-03-26 00:58:15.344813 {{(pid=105681) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1421}}
Mar 26 00:58:15.345075 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a03dfecc-e560-44c1-8a30-fa8cb3f18461 None None] OVN-Northbound DB sync process completed @ 2026-03-26 00:58:15.344901 {{(pid=105681) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
Mar 26 00:58:24.771104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-03-26 00:58:24.770803 {{(pid=105681) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1449}}
Mar 26 00:58:24.771763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] OVN-SB Sync hostname and physical networks started @ 2026-03-26 00:58:24.771633 {{(pid=105681) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1476}}
Mar 26 00:58:24.772604 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Mar 26 00:58:24.777140 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777968 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777968 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.777968 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.778145 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.778145 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.778244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.778244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.778545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.778783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.778875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.778931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.779103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.779266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.779349 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=105681) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:96}}
Mar 26 00:58:24.780477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.780792 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
Mar 26 00:58:24.780884 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
Mar 26 00:58:24.781567 np0000160849 neutron-ovn-maintenance-worker[105681]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] New host np0000160849 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
Mar 26 00:58:24.783697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.789840 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.795156 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.795930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:24.797237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.799152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.800683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:24.802895 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.805356 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9a6376f3-5fb0-425e-9698-66a297a5ec7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:24.820442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.822059 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:24.823607 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.828494 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Mar 26 00:58:24.829371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG oslo_concurrency.lockutils [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=105681) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Mar 26 00:58:24.829621 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG oslo_concurrency.lockutils [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.000s {{(pid=105681) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Mar 26 00:58:24.830757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:24.835639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.884452 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.892881 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.906361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.909615 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.926652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron_lib.callbacks.manager [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-139331', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-1621380'] for segment_host_mapping (None), after_create {{(pid=105681) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Mar 26 00:58:24.929944 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:24.935398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:24.947469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.955542 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:24.961645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.966402 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:58:24.973658 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.979655 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:24.981000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:24.993688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:24.993688 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:24.993688 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:24.993688 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:24.993688 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:24.993688 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:24.993688 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:24.993688 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:25.001100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:25.005442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.069794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.071291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:25.078210 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:25.080012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-89e16ac8-2295-4fee-9fff-003b28abefdb None None] Maintenance task: No inconsistencies found. Skipping {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:383}}
Mar 26 00:58:25.081506 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:25.083224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Maintenance task: Check missing prefix router_name in external_ids of LRPs. {{(pid=105681) update_lrouter_ports_ext_ids_name_prefix /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:500}}
Mar 26 00:58:25.083320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.084064 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.db.ovn_hash_ring_db [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Cleaned up Hash Ring nodes older than 5 days
Mar 26 00:58:25.089761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:25.095998 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Maintenance task: Check missing prefix router_name in LRPs (took 0.00 seconds)
Mar 26 00:58:25.101843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:25.103230 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.services.segments.db [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] Segments {'200618ec-0540-41b0-9429-dc7dfc74abb0'} mapped to the host np0000160849 {{(pid=105681) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
Mar 26 00:58:25.109002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:25.111857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.112060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:25.112060 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.112060 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:25.112060 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.112060 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.112060 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:25.112060 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.112060 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:25.115600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:25.116581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:25.136338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.137044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:25.137044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.137044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:25.137044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.137044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.137044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:25.137044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.137044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:25.137988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:25.137988 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.137988 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:25.137988 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.137988 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.137988 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:25.137988 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.137988 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:25.149283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:25.159162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG oslo_concurrency.lockutils [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.292s {{(pid=105681) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Mar 26 00:58:25.189924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=105681) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:673}}
Mar 26 00:58:25.204220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-5997c385-46ec-4d21-ba01-e059b350017d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.216029 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 916, in configure_nb_global
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:58:25.226436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.227293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:25.228172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.228172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:25.228172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-792e438c-6091-4191-8efd-aa0c94894621 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:25.241228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.242301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:25.242301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.242301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:25.242301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.242301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.242301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:25.242301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.242301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:25.265327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.280182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] OVN-SB Sync hostname and physical networks completed @ 2026-03-26 00:58:25.159322 {{(pid=105681) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1503}}
Mar 26 00:58:25.281324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-1fdda03c-dd16-495f-8d16-0bdb916c8630 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:58:25.304413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.305229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.306115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:25.306115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.306115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:25.307338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-7efa4d3f-713a-4440-937d-90adbedf8c73 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.310573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:25.310573 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.310573 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:25.310573 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.310573 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.310573 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:25.310573 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.310573 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:25.313603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.314475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:25.314475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.314475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:25.314475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.314475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.314475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:25.314475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.314475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:25.329280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:58:25.337767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.338496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:25.339277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.339277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:25.340415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.341238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:25.346880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.347692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:25.347692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.347692 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:25.347692 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.347692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.347692 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:25.347692 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.347692 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.350589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.351654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:25.435097 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:25.435821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.446093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.448134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9a6376f3-5fb0-425e-9698-66a297a5ec7a None None] Traceback (most recent call last):
Mar 26 00:58:25.448134 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.448134 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:25.448134 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.448134 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.448134 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:25.448134 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.448134 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-9a6376f3-5fb0-425e-9698-66a297a5ec7a None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.455449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:25.456175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:25.559703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-fdbb945e-eff7-40db-88ba-880fa2be2fa6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.584757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105681) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:710}}
Mar 26 00:58:25.728251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Mar 26 00:58:25.728603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-b6e98f07-70ef-4165-849f-80ffbcc60685 None None] OVN-Southbound DB sync process completed @ 2026-03-26 00:58:25.728332 {{(pid=105681) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1457}}
Mar 26 00:58:26.504922 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=846d4ff3-4d23-4119-85e2-189451b0355e, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:revision_number': '8', 'neutron:router_name': 'neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6'}, gateway_chassis=[fbdebcf9-bd14-4651-b533-a9f8c8fc101e], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:ac:55:2e', name='lrp-e0095b8b-a29d-49e5-a431-c122fbdf4880', networks=['172.24.5.160/24', '2001:db8::1/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=105681) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 26 00:58:26.505768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=867fd01d-5ce1-4938-8854-9e61dc870847, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac'}, gateway_chassis=[fbdebcf9-bd14-4651-b533-a9f8c8fc101e], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:ac:55:2e', name='lrp-e0095b8b-a29d-49e5-a431-c122fbdf4880', networks=['172.24.5.160/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': '01b73fe7-f890-48b6-af16-36032e15d1ef'}) old= {{(pid=105681) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 26 00:58:26.507129 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Mar 26 00:58:26.550117 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad synchronized; highest priority chassis 01b73fe7-f890-48b6-af16-36032e15d1ef
Mar 26 00:58:26.550917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'neutron:router_id': '623e19f7-2e8b-459b-a559-a52aeb1143fe'}),), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:26.551352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, chassis=01b73fe7-f890-48b6-af16-36032e15d1ef, priority=1, columns={}) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:26.551624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:26.552360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:26.552360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:26.552360 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:26.552360 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:26.552360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:26.552360 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:26.552360 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:26.552360 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event Traceback (most recent call last):
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     match.run(event, row, updates)
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 89, in run
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1639, in update_router_ha_chassis_group
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2385, in link_network_ha_chassis_group
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     next(self.gen)
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     next(self.gen)
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:26.554315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with self.create_transaction(
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     self.result = self.commit()
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     raise result.ex
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:26.555386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event 
Mar 26 00:58:26.580432 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad synchronized; highest priority chassis 01b73fe7-f890-48b6-af16-36032e15d1ef
Mar 26 00:58:26.580927 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'neutron:router_id': '623e19f7-2e8b-459b-a559-a52aeb1143fe'}),), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:26.581262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, chassis=01b73fe7-f890-48b6-af16-36032e15d1ef, priority=1, columns={}) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:26.581458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:26.582022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:26.582022 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:26.582022 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:26.582022 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:26.582022 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:26.582022 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:26.582022 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:26.582022 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event Traceback (most recent call last):
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     match.run(event, row, updates)
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 81, in run
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1639, in update_router_ha_chassis_group
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2385, in link_network_ha_chassis_group
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     next(self.gen)
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     next(self.gen)
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:26.582823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with self.create_transaction(
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     self.result = self.commit()
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     raise result.ex
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:26.583675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event 
Mar 26 00:58:29.799275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:29.801552 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:29.837786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:29.838348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.839487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:58:29.839487 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:29.839487 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:29.839487 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:29.839487 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:29.839487 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:29.839487 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.839487 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.840442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.841225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:29.907440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:29.909429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:29.909805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.910934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:29.910934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:29.910934 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:29.910934 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:29.910934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:29.910934 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:29.910934 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.910934 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.913002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.913786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:29.962642 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:29.963587 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:58:29.964358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:29.964669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.965499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:29.965499 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:29.965499 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:29.965499 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:29.965499 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:29.965499 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:29.965499 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.965499 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:29.966498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:29.967215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:29.967215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:29.967215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:29.967215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:29.967215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:29.967215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:29.967215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.967215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:58:29.967568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:29.968223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:29.968967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:29.968967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.968967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:29.995137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:29.996071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:29.996891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:29.997394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.998343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:29.998343 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:29.998343 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:29.998343 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:29.998343 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:29.998343 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:29.998343 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.998343 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:29.999795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:30.000556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:30.000556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:30.000556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:30.000556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:30.000556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:30.000556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:30.000556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:30.000556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:58:30.001449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:30.002167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:30.002878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:30.002878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:30.085787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:30.086950 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:30.087902 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:30.088378 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:30.088590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:30.089622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:30.089622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:30.089622 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:30.089622 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:30.089622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:30.089622 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:30.089622 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:30.089622 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:30.091968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:30.092696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:34.800266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:34.802535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:34.839742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:34.840004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.840548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:58:34.840548 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:34.840548 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:34.840548 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:34.840548 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:34.840548 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:34.840548 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.840548 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:34.841390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.842252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:34.908386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:34.909882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:34.910274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.911018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:34.911018 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:34.911018 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:34.911018 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:34.911018 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:34.911018 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:34.911018 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.911018 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:34.912733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.913593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:34.963431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:34.964354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:58:34.965264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:34.965622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.966722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:34.966722 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:34.966722 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:34.966722 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:34.966722 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:34.966722 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:34.966722 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.966722 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:34.968628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:34.969325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:34.969325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:34.969325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:34.969325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:34.969325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:34.969325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:34.969325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.969325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:58:34.970478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:34.971285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:34.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:34.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:34.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:34.996021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:34.996963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:34.997567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:34.997929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.000306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:35.000306 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:35.000306 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:35.000306 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:35.000306 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:35.000306 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:35.000306 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.000306 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:35.001721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:35.002484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:35.002484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:35.002484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:35.002484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:35.002484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:35.002484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:35.002484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.002484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:58:35.002865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:35.004159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:35.005294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.005294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:35.086929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:35.088094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:35.088618 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:35.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:35.089507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.090532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:35.090532 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:35.090532 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:35.090532 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:35.090532 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:35.090532 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:35.090532 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.090532 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:35.092456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:35.093585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:39.801553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:39.803879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:39.839221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:39.839483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.840165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:58:39.840165 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:39.840165 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:39.840165 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:39.840165 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:39.840165 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:39.840165 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.840165 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:39.841300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.842116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:39.909626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:39.910998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:39.911270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.912059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:39.912059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:39.912059 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:39.912059 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:39.912059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:39.912059 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:39.912059 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.912059 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:39.913648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.914424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:39.964235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:39.965180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:58:39.965688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:39.965961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.966692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:39.966692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:39.966692 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:39.966692 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:39.966692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:39.966692 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:39.966692 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.966692 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:39.968157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:39.969110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:39.969110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:39.969110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:39.969110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:39.969110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:39.969110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:39.969110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.969110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:58:39.970090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:39.971039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:39.971916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:39.971916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.971916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:39.996910 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:39.997730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:39.998312 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:39.998669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.999414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:39.999414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:39.999414 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:39.999414 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:39.999414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:39.999414 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:39.999414 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:39.999414 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:40.000768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:40.001636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:40.001636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:40.001636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:40.001636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:40.001636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:40.001636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:40.001636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:40.001636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:58:40.002423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:40.003054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:40.003746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:40.003746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:40.088144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:40.089346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:40.089765 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:40.090001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:40.090124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:40.090710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:40.090710 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:40.090710 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:40.090710 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:40.090710 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:40.090710 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:40.090710 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:40.090710 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:40.092530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:40.093416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:44.803030 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:44.805375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:44.838247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:44.838510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.839210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:58:44.839210 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:44.839210 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:44.839210 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:44.839210 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:44.839210 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:44.839210 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.839210 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:44.840136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.840667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:44.910913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:44.912795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:44.913203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.914179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:44.914179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:44.914179 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:44.914179 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:44.914179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:44.914179 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:44.914179 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.914179 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:44.915904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.916661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:44.965152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:44.966107 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:58:44.966545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:44.966915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:44.970642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.971685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:44.972351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:44.973191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:44.997806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:44.999015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:44.999623 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:44.999972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.000680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:45.000680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:45.000680 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:45.000680 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:45.000680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:45.000680 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:45.000680 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.000680 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:45.002003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:45.002772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:45.002772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:45.002772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:45.002772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:45.002772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:45.002772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:45.002772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.002772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:58:45.003684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:45.004535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:45.005316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.005316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:45.089335 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:45.090576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:45.091154 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:45.091515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:45.091689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.092447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:45.092447 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:45.092447 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:45.092447 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:45.092447 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:45.092447 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:45.092447 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.092447 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:45.094624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:45.095641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:49.804635 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:49.806709 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:49.842148 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:49.842937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.843545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:58:49.843545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:49.843545 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:49.843545 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:49.843545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:49.843545 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:49.843545 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.843545 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:49.844628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.845238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:49.912638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:49.914323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:49.914640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.915451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:49.915451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:49.915451 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:49.915451 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:49.915451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:49.915451 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:49.915451 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.915451 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:49.917107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.917475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:49.966111 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:49.967392 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:58:49.968144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:49.968513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.969509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:49.969509 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:49.969509 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:49.969509 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:49.969509 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:49.969509 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:49.969509 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.969509 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:49.970711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:49.971405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:49.971405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:49.971405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:49.971405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:49.971405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:49.971405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:49.971405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.971405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:58:49.972293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:49.972961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:49.973805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:49.973805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:49.973805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:49.999056 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:50.000359 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:50.001348 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:50.001736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.002679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:50.002679 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:50.002679 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:50.002679 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:50.002679 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:50.002679 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:50.002679 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.002679 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:50.004343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:50.005373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:50.005373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:50.005373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:50.005373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:50.005373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:50.005373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:50.005373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.005373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:58:50.006195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:50.007099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:50.008009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.008009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:50.094657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:50.094657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:50.094657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:50.094657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:50.094657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.095115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:50.095115 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:50.095115 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:50.095115 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:50.095115 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:50.095115 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:50.095115 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.095115 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:50.096568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:50.097339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:54.805941 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:54.808037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:54.841377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:54.841377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.841476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:58:54.841476 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:54.841476 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:54.841476 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:54.841476 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:54.841476 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:54.841476 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.841476 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:54.842553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.843056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:54.913839 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:54.916463 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:54.916463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.916789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:54.916789 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:54.916789 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:54.916789 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:54.916789 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:54.916789 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:54.916789 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.916789 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:54.918877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.919647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:54.967594 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:54.968347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:58:54.970149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:54.970149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.970314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:54.970314 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:54.970314 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:54.970314 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:54.970314 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:54.970314 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:54.970314 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.970314 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:54.971193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:54.971928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:54.971928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:54.971928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:54.971928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:54.971928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:54.971928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:54.971928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.971928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:58:54.972356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:54.973145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:54.974079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:54.974079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:54.974079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:55.000007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:55.000851 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.003322 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:55.004062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:55.004550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:55.004550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:55.004550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:55.004550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:55.004550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:55.004550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:55.004550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.004550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:58:55.006229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:55.006663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:55.007166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.007166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:55.091764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:55.093127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:55.095129 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:55.095129 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:55.095129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.095351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:55.095351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:55.095351 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:55.095351 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:55.095351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:55.095351 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:55.095351 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.095351 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:55.096315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:55.097129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:59.807120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:59.809244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:58:59.838118 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:59.838434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.839269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Traceback (most recent call last):
Mar 26 00:58:59.839269 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:59.839269 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:59.839269 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:59.839269 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:59.839269 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:59.839269 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.839269 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:59.841193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.842215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:59.914950 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:59.917465 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:59.917465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.917858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:59.917858 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:59.917858 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:59.917858 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:59.917858 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:59.917858 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:59.917858 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.917858 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:59.920297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.921234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:58:59.968529 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:58:59.969414 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:58:59.969852 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:58:59.970196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.970857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:58:59.970857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:59.970857 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:58:59.970857 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:58:59.970857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:59.970857 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:58:59.970857 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.970857 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:58:59.972093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:59.973017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:58:59.973017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:59.973017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:58:59.973017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:58:59.973017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:59.973017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:58:59.973017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.973017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:58:59.973590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:58:59.974470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:58:59.976133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:58:59.976133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:58:59.976133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:00.000880 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:00.001747 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:00.002369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:00.002718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.003469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:00.003469 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:00.003469 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:00.003469 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:00.003469 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:00.003469 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:00.003469 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.003469 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:00.005102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:00.006107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:00.006107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:00.006107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:00.006107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:00.006107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:00.006107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:00.006107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.006107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:00.006582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:00.007493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:00.008613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.008613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:00.093316 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:00.096125 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:00.096968 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:00.097520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:00.097842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.098803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:00.098803 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:00.098803 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:00.098803 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:00.098803 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:00.098803 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:00.098803 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.098803 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:00.101102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:00.101469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:04.808385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:04.810749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:04.848549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:04.848926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.849863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Traceback (most recent call last):
Mar 26 00:59:04.849863 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:04.849863 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:04.849863 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:04.849863 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:04.849863 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:04.849863 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.849863 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:04.851168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.851828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:04.915905 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:04.917350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:04.917730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.918618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:04.918618 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:04.918618 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:04.918618 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:04.918618 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:04.918618 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:04.918618 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.918618 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:04.920262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.920852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:04.969703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:04.970636 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:04.971236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:04.971535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.972266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:04.972266 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:04.972266 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:04.972266 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:04.972266 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:04.972266 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:04.972266 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.972266 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:04.973638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:04.974420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:04.974420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:04.974420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:04.974420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:04.974420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:04.974420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:04.974420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.974420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:04.975258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:04.976072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:04.976932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:04.976932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:04.976932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:05.002024 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:05.003080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:05.003645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:05.004182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.004947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:05.004947 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:05.004947 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:05.004947 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:05.004947 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:05.004947 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:05.004947 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.004947 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:05.006431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:05.007403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:05.007403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:05.007403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:05.007403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:05.007403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:05.007403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:05.007403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.007403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:05.008097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:05.008657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:05.009298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.009298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:05.096364 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:05.097330 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:05.098023 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:05.098557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:05.098834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.099819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:05.099819 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:05.099819 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:05.099819 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:05.099819 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:05.099819 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:05.099819 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.099819 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:05.101765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:05.102627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:09.809701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:09.811995 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:09.844119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:09.844477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.845397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:59:09.845397 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:09.845397 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:09.845397 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:09.845397 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:09.845397 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:09.845397 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.845397 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:09.846573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.847388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:09.917010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:09.918767 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:09.919183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.920090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:09.920090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:09.920090 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:09.920090 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:09.920090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:09.920090 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:09.920090 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.920090 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:09.921802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.922299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:09.971140 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:09.972390 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:09.973215 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:09.973695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.975100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:09.975100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:09.975100 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:09.975100 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:09.975100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:09.975100 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:09.975100 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.975100 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:09.976603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.977836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:09.978887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:09.980195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:10.003117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:10.004283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:10.005055 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:10.005492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.006525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:10.006525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:10.006525 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:10.006525 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:10.006525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:10.006525 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:10.006525 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.006525 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:10.007933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:10.008753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:10.008753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:10.008753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:10.008753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:10.008753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:10.008753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:10.008753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.008753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:10.009889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:10.011129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:10.012008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.012008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:10.097376 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:10.098423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:10.098925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:10.099472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:10.099685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.100462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:10.100462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:10.100462 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:10.100462 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:10.100462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:10.100462 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:10.100462 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.100462 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:10.101555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:10.102570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:14.815721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:14.815721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:14.815721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=557,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 00:59:14.854838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:14.854838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.855096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:59:14.855096 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:14.855096 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:14.855096 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:14.855096 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:14.855096 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:14.855096 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.855096 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:14.856279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.857144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:14.918339 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:14.921317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:14.921317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.921612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:14.921612 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:14.921612 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:14.921612 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:14.921612 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:14.921612 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:14.921612 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.921612 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:14.923385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.924183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:14.972472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:14.973525 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:14.974438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:14.974438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.974714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:14.974714 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:14.974714 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:14.974714 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:14.974714 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:14.974714 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:14.974714 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.974714 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:14.976330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:14.977177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:14.977177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:14.977177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:14.977177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:14.977177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:14.977177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:14.977177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.977177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:14.978389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:14.979257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:14.980153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:14.980153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:14.980153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:15.004188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:15.004946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:15.006030 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:15.006030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.007142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:15.007142 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:15.007142 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:15.007142 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:15.007142 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:15.007142 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:15.007142 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.007142 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:15.008415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:15.009506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:15.009506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:15.009506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:15.009506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:15.009506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:15.009506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:15.009506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.009506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:15.010245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:15.011238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:15.012299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.012299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:15.098726 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:15.100201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:15.101179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:15.101365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:15.101757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.102336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:15.102336 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:15.102336 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:15.102336 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:15.102336 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:15.102336 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:15.102336 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.102336 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:15.104119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:15.104487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Traceback (most recent call last):
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.862275 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:19.863671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.864489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:19.919568 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:19.920993 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:19.921204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.921855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:19.921855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:19.921855 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:19.921855 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:19.921855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:19.921855 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:19.921855 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.921855 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:19.923788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.924276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:19.973730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:19.975359 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:19.976172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:19.976511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.977357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:19.977357 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:19.977357 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:19.977357 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:19.977357 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:19.977357 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:19.977357 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.977357 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:19.978743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:19.979919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:19.979919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:19.979919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:19.979919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:19.979919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:19.979919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:19.979919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.979919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:19.980626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:19.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:19.982940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:19.982940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:19.982940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:20.005000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:20.006056 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:20.006813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:20.007529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.008572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:20.008572 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:20.008572 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:20.008572 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:20.008572 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:20.008572 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:20.008572 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.008572 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:20.010132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:20.010820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:20.010820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:20.010820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:20.010820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:20.010820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:20.010820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:20.010820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.010820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:20.012405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:20.013163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:20.014013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.014013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:20.100395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:20.101889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:20.102367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:20.102752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:20.102942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.103875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:20.103875 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:20.103875 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:20.103875 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:20.103875 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:20.103875 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:20.103875 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.103875 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:20.105443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:20.105931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:24.823409 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:24.824371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:24.857453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:24.857679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.862785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 00:59:24.862785 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:24.862785 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:24.862785 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:24.862785 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:24.862785 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:24.862785 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.862785 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:24.863927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.864486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:24.920423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:24.921723 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:24.922107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.922888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:24.922888 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:24.922888 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:24.922888 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:24.922888 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:24.922888 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:24.922888 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.922888 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:24.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.925400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:24.975284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:24.975699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:24.976104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:24.976249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.977077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:24.977077 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:24.977077 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:24.977077 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:24.977077 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:24.977077 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:24.977077 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.977077 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:24.978339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:24.979131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:24.979131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:24.979131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:24.979131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:24.979131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:24.979131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:24.979131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.979131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:24.979798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:24.980650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:24.981548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:24.981548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:24.981548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:25.005860 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:25.006472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:25.006957 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:25.007200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.007771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:25.007771 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:25.007771 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:25.007771 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:25.007771 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:25.007771 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:25.007771 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.007771 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:25.009187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:25.009871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:25.009871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:25.009871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:25.009871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:25.009871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:25.009871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:25.009871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.009871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:25.011250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:25.012189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:25.013356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.013356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:25.101867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:25.102324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:25.102762 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:25.103206 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:25.103206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.103587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:25.103587 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:25.103587 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:25.103587 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:25.103587 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:25.103587 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:25.103587 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.103587 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:25.105565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:25.106049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:29.823059 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:29.824953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:29.861448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:29.862078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.863474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 00:59:29.863474 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:29.863474 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:29.863474 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:29.863474 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:29.863474 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:29.863474 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.863474 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:29.864880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.866183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:29.921391 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:29.922834 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:29.923353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.924443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:29.924443 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:29.924443 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:29.924443 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:29.924443 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:29.924443 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:29.924443 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.924443 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:29.926749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.927676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:29.975850 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:29.976402 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:29.977185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:29.977578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.978428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:29.978428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:29.978428 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:29.978428 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:29.978428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:29.978428 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:29.978428 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.978428 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:29.979193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.979897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:29.980603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:29.981447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:30.006511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:30.007011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:30.007651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:30.007916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.008683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:30.008683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:30.008683 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:30.008683 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:30.008683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:30.008683 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:30.008683 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.008683 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:30.010491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:30.011494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:30.011494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:30.011494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:30.011494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:30.011494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:30.011494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:30.011494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.011494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:30.012855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:30.013730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:30.014597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.014597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:30.102575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:30.104073 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:30.104540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:30.104762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.105954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:30.105954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:30.105954 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:30.105954 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:30.105954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:30.105954 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:30.105954 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.105954 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:30.108025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:30.108551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:34.823942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:34.826137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36a48f56-61bd-4a0f-9dd4-820437bbe49b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:34.858432 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:34.858721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.859409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36a48f56-61bd-4a0f-9dd4-820437bbe49b None None] Traceback (most recent call last):
Mar 26 00:59:34.859409 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:34.859409 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:34.859409 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:34.859409 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:34.859409 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:34.859409 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.859409 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-36a48f56-61bd-4a0f-9dd4-820437bbe49b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:34.860780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.861332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:34.922144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:34.923615 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:34.923922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:34.924911 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:34.924911 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:34.924911 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:34.924911 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:34.924911 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:34.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.924911 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:34.926800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.928114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:34.976424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:34.977237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:34.977971 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:34.978391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.979232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:34.979232 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:34.979232 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:34.979232 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:34.979232 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:34.979232 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:34.979232 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.979232 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:34.980661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:34.981930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:34.981930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:34.981930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:34.981930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:34.981930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:34.981930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:34.981930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.981930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:34.982872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:34.983730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:34.984751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:34.984751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:34.984751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:35.007410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:35.007866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:35.008580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:35.009048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.009866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:35.009866 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:35.009866 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:35.009866 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:35.009866 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:35.009866 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:35.009866 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.009866 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:35.011172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:35.011885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:35.011885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:35.011885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:35.011885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:35.011885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:35.011885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:35.011885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.011885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:35.012712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:35.013495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:35.014343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.014343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:35.103378 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:35.103838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:35.104279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:35.104553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:35.104687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.105361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:35.105361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:35.105361 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:35.105361 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:35.105361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:35.105361 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:35.105361 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.105361 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:35.107018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:35.107500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:39.824619 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:39.826255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:39.860221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:39.860688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.861567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 00:59:39.861567 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:39.861567 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:39.861567 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:39.861567 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:39.861567 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:39.861567 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.861567 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:39.862858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.863669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:39.922953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:39.924332 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:39.924611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.925325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:39.925325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:39.925325 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:39.925325 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:39.925325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:39.925325 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:39.925325 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.925325 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:39.926263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.927138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:39.977380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:39.977888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:39.978770 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:39.979139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.980074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:39.980074 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:39.980074 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:39.980074 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:39.980074 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:39.980074 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:39.980074 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.980074 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:39.981349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:39.982112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:39.982112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:39.982112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:39.982112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:39.982112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:39.982112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:39.982112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.982112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:39.982966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:39.983758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:39.984601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:39.984601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:39.984601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:40.008016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:40.008309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:40.008679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:40.008910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.009406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:40.009406 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:40.009406 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:40.009406 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:40.009406 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:40.009406 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:40.009406 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.009406 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:40.010176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:40.010910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:40.010910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:40.010910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:40.010910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:40.010910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:40.010910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:40.010910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.010910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:40.011282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:40.012097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:40.012905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.012905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:40.104106 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:40.104556 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:40.105137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:40.105575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:40.105755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.106557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:40.106557 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:40.106557 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:40.106557 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:40.106557 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:40.106557 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:40.106557 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.106557 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:40.108326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:40.109100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:44.825369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:44.827091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5160a8fc-d61d-4dd5-b084-bc0773112903 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:44.866374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:44.866763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.868084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5160a8fc-d61d-4dd5-b084-bc0773112903 None None] Traceback (most recent call last):
Mar 26 00:59:44.868084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:44.868084 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:44.868084 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:44.868084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:44.868084 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:44.868084 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.868084 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-5160a8fc-d61d-4dd5-b084-bc0773112903 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:44.869569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:44.924098 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:44.925430 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:44.925800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.926877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:44.926877 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:44.926877 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:44.926877 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:44.926877 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:44.926877 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:44.926877 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.926877 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:44.929014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.929840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:44.978057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:44.978523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:44.979605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:44.979927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.980920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:44.980920 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:44.980920 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:44.980920 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:44.980920 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:44.980920 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:44.980920 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.980920 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:44.982426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:44.983270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:44.983270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:44.983270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:44.983270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:44.983270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:44.983270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:44.983270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.983270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:44.983928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:44.984915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:44.985919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:44.985919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:44.985919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:45.008728 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:45.009279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:45.010397 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:45.010765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.011728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:45.011728 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:45.011728 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:45.011728 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:45.011728 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:45.011728 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:45.011728 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.011728 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:45.014366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:45.014927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:45.014927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:45.014927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:45.014927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:45.014927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:45.014927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:45.014927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.014927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:45.017070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:45.017606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:45.018174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.018174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:45.104579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:45.104848 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:45.105506 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:45.105886 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:45.106159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.106878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:45.106878 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:45.106878 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:45.106878 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:45.106878 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:45.106878 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:45.106878 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.106878 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:45.107861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:45.108355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:49.826203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:49.829755 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:49.863791 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:49.864435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.865183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Traceback (most recent call last):
Mar 26 00:59:49.865183 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:49.865183 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:49.865183 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:49.865183 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:49.865183 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:49.865183 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.865183 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-79ad5cc2-46cb-4095-91ff-ebac54b928fe None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:49.866715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.867569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:49.924774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:49.925996 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:49.926471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.927404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:49.927404 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:49.927404 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:49.927404 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:49.927404 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:49.927404 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:49.927404 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.927404 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:49.929220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.930069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:49.978722 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:49.979330 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:49.980132 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:49.980613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.981597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:49.981597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:49.981597 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:49.981597 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:49.981597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:49.981597 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:49.981597 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.981597 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:49.983415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:49.984261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:49.984261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:49.984261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:49.984261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:49.984261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:49.984261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:49.984261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.984261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:49.985154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:49.986149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:49.987136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:49.987136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:49.987136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:50.009223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:50.009847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:50.010430 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:50.010705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:50.014061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.014961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:50.016198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:50.017004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:50.017834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.017834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:50.105023 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:50.105291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:50.106099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:50.106534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:50.106740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.107587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:50.107587 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:50.107587 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:50.107587 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:50.107587 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:50.107587 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:50.107587 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.107587 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:50.108714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:50.109612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:54.828410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:54.829392 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:54.866607 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.873743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:54.874206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:54.874585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:54.874585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:54.874585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:54.874585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:54.874585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:54.874585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.874585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:54.925288 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:54.926513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:54.926841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.927707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:54.927707 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:54.927707 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:54.927707 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:54.927707 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:54.927707 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:54.927707 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.927707 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:54.929428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.930619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:54.979239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:54.979858 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:54.980549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:54.980860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.981643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:54.981643 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:54.981643 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:54.981643 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:54.981643 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:54.981643 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:54.981643 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.981643 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:54.982942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:54.984175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:54.984175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:54.984175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:54.984175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:54.984175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:54.984175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:54.984175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.984175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:54.984791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:54.985492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:54.986600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:54.986600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:54.986600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:55.009832 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:55.010349 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:55.015179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:55.015577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.016301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:55.016301 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:55.016301 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:55.016301 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:55.016301 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:55.016301 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:55.016301 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.016301 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:55.017884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:55.019135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:55.019135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:55.019135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:55.019135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:55.019135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:55.019135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:55.019135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.019135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 00:59:55.020883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:55.022407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:55.023394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.023394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:55.105628 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.126475 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:55.127770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:55.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:59.829306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:59.830907 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 00:59:59.866833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:59.867323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.868255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 00:59:59.868255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:59.868255 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:59.868255 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:59.868255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:59.868255 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:59.868255 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.868255 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:59.869267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.870416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:59.925961 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:59.927106 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:59.927386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.928374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:59.928374 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:59.928374 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:59.928374 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:59.928374 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:59.928374 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:59.928374 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.928374 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:59.930503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.931760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 00:59:59.979836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 00:59:59.980448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 00:59:59.981395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 00:59:59.981754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.982686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 00:59:59.982686 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:59.982686 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 00:59:59.982686 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 00:59:59.982686 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:59.982686 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 00:59:59.982686 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.982686 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 00:59:59.984217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:59.985848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 00:59:59.985848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:59.985848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 00:59:59.985848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 00:59:59.985848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:59.985848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 00:59:59.985848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.985848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 00:59:59.986304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 00:59:59.987373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 00:59:59.988963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 00:59:59.988963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 00:59:59.988963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:00.010395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:00.010841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:00.011475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:00.011873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.012730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:00.012730 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:00.012730 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:00.012730 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:00.012730 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:00.012730 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:00.012730 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.012730 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:00.014175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:00.014951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:00.014951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:00.014951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:00.014951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:00.014951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:00.014951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:00.014951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.014951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:00.015676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:00.016563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:00.017534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.017534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:00.106620 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:00.107217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:00.107651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:00.108057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:00.108241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.109002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:00.109002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:00.109002 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:00.109002 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:00.109002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:00.109002 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:00.109002 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.109002 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:00.110583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:00.111063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:04.829905 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:04.831550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:04.864853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:04.865409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.866453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:00:04.866453 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:04.866453 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:04.866453 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:04.866453 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:04.866453 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:04.866453 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.866453 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:04.867782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.868601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:04.926583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:04.927535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:04.927798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.928585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:04.928585 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:04.928585 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:04.928585 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:04.928585 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:04.928585 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:04.928585 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.928585 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:04.930301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.931085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:04.980305 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:04.980544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:04.981284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:04.981614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.982460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:04.982460 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:04.982460 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:04.982460 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:04.982460 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:04.982460 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:04.982460 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.982460 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:04.983508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:04.984006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:04.984006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:04.984006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:04.984006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:04.984006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:04.984006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:04.984006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.984006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:04.984503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:04.984963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:04.985554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:04.985554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:04.985554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:05.010816 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:05.011219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:05.011633 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:05.011933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.012490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:05.012490 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:05.012490 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:05.012490 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:05.012490 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:05.012490 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:05.012490 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.012490 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:05.013289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:05.014003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:05.014003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:05.014003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:05.014003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:05.014003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:05.014003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:05.014003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.014003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:05.014535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:05.015371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:05.016417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.016417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:05.107176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:05.107539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:05.108130 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:05.108499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:05.108684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.109452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:05.109452 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:05.109452 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:05.109452 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:05.109452 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:05.109452 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:05.109452 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.109452 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:05.111132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:05.111932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:09.830626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:09.831868 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:09.868781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:09.868781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.869151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:00:09.869151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:09.869151 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:09.869151 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:09.869151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:09.869151 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:09.869151 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.869151 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:09.870403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.871321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:09.927249 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:09.929094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:09.929094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.929326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:09.929326 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:09.929326 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:09.929326 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:09.929326 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:09.929326 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:09.929326 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.929326 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:09.930480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.931309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:09.981062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:09.981267 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:09.982935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:09.982935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.983157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:09.983157 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:09.983157 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:09.983157 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:09.983157 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:09.983157 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:09.983157 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.983157 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:09.984469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:09.984955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:09.984955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:09.984955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:09.984955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:09.984955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:09.984955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:09.984955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.984955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:09.985449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:09.986021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:09.986623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:09.986623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:09.986623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:10.011277 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:10.012004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:10.012522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:10.013077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.013945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:10.013945 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:10.013945 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:10.013945 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:10.013945 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:10.013945 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:10.013945 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.013945 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:10.018021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:10.019124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:10.020351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.020870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:10.108213 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:10.108656 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:10.111142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:10.111142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:10.111142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.112108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:10.112108 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:10.112108 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:10.112108 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:10.112108 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:10.112108 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:10.112108 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.112108 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:10.113402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:10.113924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:14.831207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:14.832643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:14.846687 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=244 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:00:14.868045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:14.868325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.868890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:00:14.868890 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:14.868890 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:14.868890 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:14.868890 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:14.868890 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:14.868890 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.868890 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:14.870033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.870796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:14.928007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:14.929679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:14.930079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.931140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:14.931140 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:14.931140 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:14.931140 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:14.931140 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:14.931140 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:14.931140 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.931140 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:14.933187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.934052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:14.981576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:14.981964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:14.982717 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:14.983231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.983879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:14.983879 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:14.983879 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:14.983879 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:14.983879 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:14.983879 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:14.983879 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.983879 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:14.985464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:14.986172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:14.986172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:14.986172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:14.986172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:14.986172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:14.986172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:14.986172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.986172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:14.987004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:14.987668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:14.988635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:14.988635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:14.988635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:15.012007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:15.012464 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:15.013086 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:15.013456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.014360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:15.014360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:15.014360 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:15.014360 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:15.014360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:15.014360 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:15.014360 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.014360 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:15.015614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:15.016570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:15.016570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:15.016570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:15.016570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:15.016570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:15.016570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:15.016570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.016570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:15.017257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:15.017829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:15.018309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.018309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:15.108957 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:15.110566 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:15.111387 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:15.111784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:15.112242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.113014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:15.113014 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:15.113014 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:15.113014 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:15.113014 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:15.113014 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:15.113014 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.113014 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:15.114755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:15.115895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:19.831823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:19.833405 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:19.867465 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:19.867776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.868415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:00:19.868415 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:19.868415 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:19.868415 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:19.868415 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:19.868415 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:19.868415 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.868415 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:19.869620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.870553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:19.928729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:19.930052 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:19.930417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.931465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:19.931465 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:19.931465 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:19.931465 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:19.931465 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:19.931465 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:19.931465 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.931465 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:19.933246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.934250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:19.982211 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:19.982826 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:19.983403 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:19.983763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.984517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:19.984517 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:19.984517 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:19.984517 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:19.984517 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:19.984517 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:19.984517 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.984517 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:19.985845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:19.986397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:19.986397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:19.986397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:19.986397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:19.986397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:19.986397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:19.986397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.986397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:19.987607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:19.988124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:19.988601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:19.988601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:19.988601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:20.012517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:20.013261 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:20.014167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:20.014766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.015794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:20.015794 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:20.015794 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:20.015794 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:20.015794 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:20.015794 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:20.015794 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.015794 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:20.017470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:20.018378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:20.018378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:20.018378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:20.018378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:20.018378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:20.018378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:20.018378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.018378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:20.019355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:20.019889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:20.020804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.020804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:20.110637 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:20.111239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:20.112002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:20.112448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:20.112651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.113516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:20.113516 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:20.113516 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:20.113516 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:20.113516 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:20.113516 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:20.113516 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.113516 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:20.115447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:20.116373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:24.832417 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:24.833823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:24.867148 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:24.867425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.868108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:00:24.868108 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:24.868108 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:24.868108 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:24.868108 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:24.868108 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:24.868108 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.868108 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:24.868949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.870117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:24.929396 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:24.930497 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:24.930836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.932213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:24.932213 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:24.932213 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:24.932213 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:24.932213 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:24.932213 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:24.932213 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.932213 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:24.933530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.934424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:24.982826 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:24.983252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:24.983622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:24.983844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.984431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:24.984431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:24.984431 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:24.984431 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:24.984431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:24.984431 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:24.984431 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.984431 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:24.985256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:24.985968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:24.985968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:24.985968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:24.985968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:24.985968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:24.985968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:24.985968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.985968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:24.986819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:24.987543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:24.988402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:24.988402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:24.988402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:25.013078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:25.013314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:25.013852 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:25.014355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.015213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:25.015213 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:25.015213 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:25.015213 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:25.015213 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:25.015213 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:25.015213 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.015213 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:25.016100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:25.016963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:25.016963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:25.016963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:25.016963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:25.016963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:25.016963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:25.016963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.016963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:25.017411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:25.018148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:25.018993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.018993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:25.111249 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:25.111892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:25.112649 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:25.113097 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:25.113292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.114137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:25.114137 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:25.114137 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:25.114137 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:25.114137 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:25.114137 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:25.114137 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.114137 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:25.115767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:25.116837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:29.833074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:29.834963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:30.010090 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:30.013209 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:30.013336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:30.013967 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:30.014928 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:30.016678 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:30.017098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.018406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:30.018406 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.018406 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:30.018406 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.018406 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.018406 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:30.018406 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.018406 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.020310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.021278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:30.022378 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:30.022655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.023372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:30.023825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:30.023825 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.023825 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:30.023825 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.023825 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.023825 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:30.023825 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.023825 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:30.024882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:30.026126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:30.026831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:30.026831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.026831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:30.026831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.026831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.026831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:30.026831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.026831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:30.027845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.028385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.028939 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:30.032943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:30.033445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:30.033445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.033445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:30.033445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.033445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.033445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:30.033445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.033445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:30.036907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.037442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:30.038387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.038387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:30.054998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:30.055545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.056665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:00:30.056665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.056665 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:30.056665 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.056665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.056665 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:30.056665 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.056665 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.057720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.058308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:30.112183 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:30.112919 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:30.113961 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:30.114470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:30.114677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.115717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:30.115717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.115717 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:30.115717 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.115717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.115717 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:30.115717 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.115717 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.117623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:30.118206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:34.834189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:34.835969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:34.869265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:34.869966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:34.870883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:00:34.870883 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:34.870883 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:34.870883 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:34.870883 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:34.870883 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:34.870883 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:34.870883 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:34.872211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:34.873089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:35.014363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:35.014699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:35.015404 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:35.016289 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:35.017295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:35.017654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:35.017937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.018913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:35.019423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.020362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:35.020362 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.020362 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:35.020362 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.020362 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.020362 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:35.020362 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.020362 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:35.020912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:35.020912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.020912 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:35.020912 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.020912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.020912 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:35.020912 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.020912 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:35.021329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.023877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.024597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:35.025536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:35.026075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:35.028147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.028603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.029162 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:35.030096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:35.030561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:35.030561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.030561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:35.030561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.030561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.030561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:35.030561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.030561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:35.031090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.031671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.032457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:35.032457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.032457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:35.113474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:35.114407 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:35.115083 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:35.115464 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:35.115659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.116550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:35.116550 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.116550 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:35.116550 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.116550 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.116550 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:35.116550 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.116550 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.118775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:35.119828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:39.834873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:39.836429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:39.866347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:39.866729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:39.867659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:00:39.867659 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:39.867659 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:39.867659 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:39.867659 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:39.867659 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:39.867659 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:39.867659 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:39.868829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:39.869423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:40.015462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:40.016585 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:40.017327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:40.018042 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:40.018351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:40.018796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.019117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:40.019856 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:40.020478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:40.020478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.020478 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:40.020478 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.020478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.020478 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:40.020478 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.020478 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:40.021050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.022723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:40.023938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:40.025135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.026145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:40.027354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:40.028451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.029044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.029518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.029942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:40.030421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:40.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:40.115160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:40.115160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:40.115827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:40.116244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:40.116489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.117527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:40.117527 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.117527 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:40.117527 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.117527 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.117527 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:40.117527 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.117527 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.119268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:40.120158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:44.835594 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:44.837133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:44.870799 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:44.871235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:44.872155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:00:44.872155 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:44.872155 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:44.872155 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:44.872155 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:44.872155 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:44.872155 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:44.872155 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:44.873110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:44.874211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:45.016421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:45.017510 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:45.018045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:45.018483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.019102 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:45.019381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:45.020502 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:45.020943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:45.020943 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.020943 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:45.020943 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.020943 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.020943 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:45.020943 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.020943 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.023406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.023932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:45.024470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:45.024470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.025177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:45.025177 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.025177 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:45.025177 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.025177 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.025177 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:45.025177 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.025177 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:45.027096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:45.027573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:45.029420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.029895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.030437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:45.030437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.030437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:45.030437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.030711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:45.030711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.030711 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:45.030711 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.030711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.030711 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:45.030711 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.030711 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:45.032129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:45.032558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:45.032558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.032558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:45.032558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.032558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.032558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:45.032558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.032558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:45.033255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.033702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:45.034205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.034205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:45.115272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:45.115839 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:45.116742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:45.117313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:45.117572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.118535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:45.118535 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.118535 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:45.118535 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.118535 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.118535 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:45.118535 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.118535 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.119732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:45.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:49.836452 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:49.838057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:49.871633 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:49.872063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:49.873135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:00:49.873135 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:49.873135 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:49.873135 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:49.873135 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:49.873135 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:49.873135 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:49.873135 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:49.874309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:49.875294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:50.017100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:50.018534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:50.019054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.020008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:50.020578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:50.020578 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.020578 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:50.020578 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.020578 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.020578 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:50.020578 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.020578 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:50.021207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:50.021523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.023768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.024147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:50.024486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:50.024713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:50.025082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.025957 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:50.026492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:50.026492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.026492 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:50.026492 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.026492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.026492 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:50.026492 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.026492 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:50.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:50.028529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.029411 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:50.030766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.031545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:50.032380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.033087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:50.033549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.034340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:50.035169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.035169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:50.115924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:50.116768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:50.117475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:50.117972 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:50.118481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.119561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:50.119561 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.119561 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:50.119561 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.119561 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.119561 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:50.119561 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.119561 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.121601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:50.122886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:54.836941 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:54.838736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:54.881511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:54.881864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:54.882820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:00:54.882820 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:54.882820 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:54.882820 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:54.882820 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:54.882820 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:54.882820 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:54.882820 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:54.884135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:54.885012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:55.017821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:55.077746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:55.079629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:55.081147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:55.082102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:55.082892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.084358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:55.085162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.085920 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:00:55.086856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.088220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:55.089685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.090755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:55.117026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:55.117561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:55.117879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:55.118083 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:55.118185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.118605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:00:55.118605 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.118605 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:55.118605 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.118605 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.118605 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:55.118605 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.118605 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.119387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:55.120491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:00:59.837817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:00:59.839579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:00:59.878527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:00:59.879005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:59.879945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:00:59.879945 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:59.879945 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:00:59.879945 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:00:59.879945 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:59.879945 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:00:59.879945 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:59.879945 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:59.881371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:00:59.882345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:00.018563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:00.019661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:00.019951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.020767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:00.020767 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.020767 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:00.020767 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.020767 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.020767 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:00.020767 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.020767 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.022671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.023596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:00.024323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:00.024323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:00.024323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:00.024693 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:00.025352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:00.025669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.026447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:00.026447 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.026447 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:00.026447 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.026447 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.026447 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:00.026447 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.026447 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:00.028434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:00.029280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:00.030378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.031453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.032028 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:00.032338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:00.032737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:00.032737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.032737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:00.032737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.032737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.032737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:00.032737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.032737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:00.033278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.033736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:00.034232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.034232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:00.118124 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:00.118682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:00.119356 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:00.119738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:00.119890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.120756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:00.120756 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.120756 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:00.120756 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.120756 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.120756 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:00.120756 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.120756 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.122367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:00.123240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:04.838571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:04.840360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:04.877871 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:04.878257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:04.879231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:01:04.879231 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:04.879231 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:04.879231 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:04.879231 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:04.879231 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:04.879231 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:04.879231 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:04.880936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:04.882040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:05.019482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:05.020691 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:05.021079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.022082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:05.022082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.022082 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:05.022082 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.022082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.022082 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:05.022082 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.022082 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.024148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.025013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:05.025759 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:05.025759 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:05.025759 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:05.026445 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:05.026884 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:05.027246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.028235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:05.028235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.028235 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:05.028235 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.028235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.028235 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:05.028235 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.028235 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:05.029624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:05.030078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:05.032696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.033177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.033717 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:05.035161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:05.035606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:05.035606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.035606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:05.035606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.035606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.035606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:05.035606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.035606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:05.037045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.037610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:05.038276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.038276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:05.119068 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:05.119723 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:05.120292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:05.120681 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:05.120848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:05.121642 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.121642 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:05.121642 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.121642 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.121642 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:05.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.123660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:05.124555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:09.839379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:09.841098 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:09.873234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:09.873491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:09.874031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:01:09.874031 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:09.874031 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:09.874031 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:09.874031 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:09.874031 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:09.874031 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:09.874031 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:09.875241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:09.876332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:10.020162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:10.021173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:10.021506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.022408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:10.022408 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.022408 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:10.022408 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.022408 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.022408 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:10.022408 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.022408 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.024058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.024939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:10.025812 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:10.026234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:10.026656 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:10.027672 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:10.028858 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:10.029273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.030582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:10.030582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.030582 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:10.030582 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.030582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.030582 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:10.030582 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.030582 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:10.033057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:10.033999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:10.033999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.033999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:10.033999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.033999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.033999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:10.033999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.033999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:10.034287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:10.040814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.041490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.041886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:10.041886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.041886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:10.041886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.042162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:10.042162 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.042162 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:10.042162 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.042162 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.042162 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:10.042162 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.042162 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:10.043858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:10.044344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:10.044344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.044344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:10.044344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.044344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.044344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:10.044344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.044344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:10.045310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.045847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:10.046506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.046506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:10.120232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:10.121171 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:10.121890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:10.122468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:10.122602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.123491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:10.123491 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.123491 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:10.123491 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.123491 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.123491 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:10.123491 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.123491 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.125886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:10.127116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:14.840013 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:14.840937 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:14.852149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:01:14.875472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:14.875916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:14.876912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:01:14.876912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:14.876912 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:14.876912 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:14.876912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:14.876912 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:14.876912 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:14.876912 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:14.878155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:14.879311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:15.020833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:15.021841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:15.022567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.023310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:15.023310 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.023310 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:15.023310 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.023310 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.023310 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:15.023310 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.023310 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.025010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.026249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:15.027680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:15.027680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:15.027680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:15.027680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.028245 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:15.028872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:15.028872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.028872 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:15.028872 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.028872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.028872 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:15.028872 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.028872 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:15.030274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:15.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:15.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:15.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:15.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.030917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:15.031177 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:15.032938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.033929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.034669 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:15.035380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:15.036320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:15.036320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.036320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:15.036320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.036320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.036320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:15.036320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.036320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:15.036889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.037526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:15.038026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.038026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:15.120930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:15.121347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:15.122110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:15.122580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:15.122844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.123671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:15.123671 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.123671 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:15.123671 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.123671 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.123671 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:15.123671 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.123671 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.125342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:15.126139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:19.840694 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:19.841706 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:19.871181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:19.871601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:19.872423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:01:19.872423 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:19.872423 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:19.872423 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:19.872423 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:19.872423 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:19.872423 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:19.872423 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:19.873575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:19.874481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:20.021493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:20.022651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:20.022932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.023804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:20.023804 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.023804 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:20.023804 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.023804 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.023804 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:20.023804 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.023804 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.025486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.026297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:20.027146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:20.027146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:20.027663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:20.027946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.028665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:20.028665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.028665 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:20.028665 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.028665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.028665 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:20.028665 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.028665 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:20.029081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:20.030606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:20.031182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:20.031182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.031182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:20.031182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.031182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.031182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:20.031182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.031182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:20.032662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.033191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.033754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:20.033754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.033754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:20.033754 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:20.033754 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:20.033937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.034619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:20.034619 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.034619 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:20.034619 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.034619 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.034619 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:20.034619 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.034619 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:20.035999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:20.036481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:20.036481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.036481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:20.036481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.036481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.036481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:20.036481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.036481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:20.038104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.038657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:20.039336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.039336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:20.121436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:20.121893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:20.122478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:20.122838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:20.123072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.123831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:20.123831 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.123831 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:20.123831 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.123831 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.123831 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:20.123831 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.123831 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.124996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:20.125757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:24.841160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:24.842890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:24.877871 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:24.878353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:24.879379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:01:24.879379 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:24.879379 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:24.879379 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:24.879379 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:24.879379 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:24.879379 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:24.879379 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:24.880623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:24.881284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:25.022142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:25.023263 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:25.023569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.024492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:25.024492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.024492 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:25.024492 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.024492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.024492 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:25.024492 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.024492 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.026467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.027220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:25.027903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:25.028021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:25.028439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:25.028733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.029456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:25.029456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.029456 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:25.029456 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.029456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.029456 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:25.029456 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.029456 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:25.030688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:25.031359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:25.031359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.031359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:25.031359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.031359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.031359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:25.031359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.031359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:25.032356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.033019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.033808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:25.033808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.033808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:25.033808 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:25.033808 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:25.033808 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:25.034113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.034586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:25.034586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.034586 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:25.034586 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.034586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.034586 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:25.034586 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.034586 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:25.035433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:25.036093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:25.036093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.036093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:25.036093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.036093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.036093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:25.036093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.036093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:25.036490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.037142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:25.037796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.037796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:25.122120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:25.122746 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:25.123384 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:25.123650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:25.123761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.124489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:25.124489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.124489 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:25.124489 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.124489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.124489 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:25.124489 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.124489 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.126327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:25.127482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:29.841929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:29.843650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:29.881382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:29.881787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:29.882741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:01:29.882741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:29.882741 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:29.882741 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:29.882741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:29.882741 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:29.882741 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:29.882741 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:29.884268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:29.885692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:30.022798 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:30.023792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:30.024082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.024791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:30.024791 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.024791 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:30.024791 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.024791 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.024791 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:30.024791 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.024791 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.027000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.027851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:30.028806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:30.028806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:30.029039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:30.029326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.029886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:30.029886 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.029886 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:30.029886 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.029886 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.029886 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:30.029886 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.029886 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:30.031149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:30.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:30.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:30.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:30.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:30.032786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.033390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.034119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:30.034119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.034119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:30.034119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:30.034119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:30.034119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:30.034119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.034754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:30.034754 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.034754 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:30.034754 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.034754 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.034754 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:30.034754 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.034754 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:30.036259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:30.036757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:30.036757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.036757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:30.036757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.036757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.036757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:30.036757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.036757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:30.037511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.038063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:30.038614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.038614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:30.122778 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:30.123226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:30.124557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:30.124926 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:30.125769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.125923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:30.125923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.125923 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:30.125923 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.125923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.125923 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:30.125923 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.125923 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.127274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:30.127825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:34.842816 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:34.844624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:34.878741 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:34.878916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:34.879365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:01:34.879365 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:34.879365 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:34.879365 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:34.879365 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:34.879365 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:34.879365 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:34.879365 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:34.880384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:34.881586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:35.023329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:35.024674 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:35.024996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.026006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:35.026006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.026006 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:35.026006 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.026006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.026006 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:35.026006 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.026006 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.027886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.029542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:35.030388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:35.030388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:35.030388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:35.030388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.030834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:35.030834 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.030834 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:35.030834 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.030834 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.030834 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:35.030834 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.030834 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:35.033514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.034020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:35.034445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:35.034966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:35.035363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:35.035363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.036034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:35.036034 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.036034 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:35.036034 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.036034 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.036034 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:35.036034 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.036034 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:35.037464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:35.037894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:35.037894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.037894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:35.037894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.037894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.037894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:35.037894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.037894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:35.039367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.039854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:35.040387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.040387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:35.123111 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:35.124005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:35.124790 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:35.125315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:35.125540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.126504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:35.126504 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.126504 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:35.126504 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.126504 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.126504 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:35.126504 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.126504 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.128300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:35.128805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:39.843535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:39.845318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:39.900776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:39.901208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:39.902046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:01:39.902046 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:39.902046 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:39.902046 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:39.902046 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:39.902046 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:39.902046 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:39.902046 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:39.903050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:39.904038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:40.024081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:40.025249 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:40.025569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.026412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:40.026412 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.026412 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:40.026412 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.026412 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.026412 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:40.026412 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.026412 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.028223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.029139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:40.029847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:40.029847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:40.030089 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:40.030370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.031201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:40.031201 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.031201 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:40.031201 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.031201 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.031201 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:40.031201 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.031201 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:40.032456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:40.032961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:40.032961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.032961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:40.032961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.032961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.032961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:40.032961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.032961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:40.033952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.034370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.034857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:40.034857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.034857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:40.034857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:40.034857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:40.035027 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:40.035151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.035645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:01:40.035645 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.035645 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:40.035645 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.035645 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.035645 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:40.035645 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.035645 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:40.036544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.036931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:40.037830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.038293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:40.038812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.038812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.153936 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:40.154592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:40.155379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:44.844149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:44.845725 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:44.879777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:44.880100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:44.880549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:01:44.880549 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:44.880549 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:44.880549 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:44.880549 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:44.880549 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:44.880549 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:44.880549 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:44.881636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:44.882572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:45.024579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:45.025657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:45.026033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.026988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:45.026988 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.026988 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:45.026988 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.026988 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.026988 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:45.026988 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.026988 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.028760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.029336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:45.029859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:45.029859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:45.030081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:45.030451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.031136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:45.031136 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.031136 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:45.031136 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.031136 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.031136 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:45.031136 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.031136 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:45.031790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.032552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:45.033270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.034105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:45.034697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:45.034697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:45.034965 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:45.035211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.035638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:45.035638 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.035638 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:45.035638 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.035638 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.035638 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:45.035638 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.035638 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:45.036523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:45.037292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:45.037292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.037292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:45.037292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.037292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.037292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:45.037292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.037292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:45.038192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.039111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:45.039745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.039745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:45.124313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:45.124869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:45.125530 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:45.125950 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:45.126217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.127129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:45.127129 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.127129 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:45.127129 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.127129 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.127129 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:45.127129 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.127129 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.128777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:45.129621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:49.844706 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:49.846953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:49.880273 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:49.880488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:49.881104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:01:49.881104 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:49.881104 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:49.881104 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:49.881104 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:49.881104 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:49.881104 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:49.881104 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:49.882454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:49.883361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:50.025173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:50.026401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:50.027043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.027812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:50.027812 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.027812 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:50.027812 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.027812 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.027812 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:50.027812 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.027812 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.029505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.030004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:50.030474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:50.030474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:50.030834 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:50.031159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.031818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:50.031818 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.031818 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:50.031818 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.031818 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.031818 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:50.031818 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.031818 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:50.033116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:50.033618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:50.033618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.033618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:50.033618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.033618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.033618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:50.033618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.033618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:50.034856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.035336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.035787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:50.035787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.035787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:50.035787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:50.035787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:50.036144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:50.036356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.036891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:50.036891 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.036891 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:50.036891 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.036891 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.036891 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:50.036891 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.036891 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:50.038228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:50.038666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:50.038666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.038666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:50.038666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.038666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.038666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:50.038666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.038666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:50.039828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.040259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:50.040702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.040702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:50.124825 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:50.125273 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:50.125811 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:50.126257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:50.126441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.127217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:50.127217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.127217 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:50.127217 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.127217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.127217 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:50.127217 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.127217 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.129351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:50.130232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:54.847031 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:54.847913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:54.884688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:54.884953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:54.885476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:01:54.885476 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:54.885476 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:54.885476 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:54.885476 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:54.885476 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:54.885476 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:54.885476 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:54.886637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:54.887491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:55.025753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:55.026995 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:55.027404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.028410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:55.028410 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.028410 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:55.028410 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.028410 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.028410 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:55.028410 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.028410 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.030203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.031079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:55.031882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:55.031882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:01:55.031882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:55.032067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.032512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:55.032512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.032512 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:55.032512 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.032512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.032512 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:55.032512 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.032512 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:55.033756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.035362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:55.036137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:55.036872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.037431 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:01:55.038001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:55.038785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:01:55.038785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.038785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:01:55.038785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.038785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.038785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:01:55.038785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.038785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:01:55.039144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.040085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:55.040593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.040593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:55.125380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:55.125875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:55.126553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:55.127080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:55.127304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.128127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:01:55.128127 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.128127 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:55.128127 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.128127 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.128127 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:55.128127 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.128127 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.129990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:55.130805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:01:59.846081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:01:59.847563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:01:59.879880 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:01:59.880174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:59.880847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:01:59.880847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:59.880847 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:01:59.880847 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:01:59.880847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:59.880847 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:01:59.880847 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:59.880847 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:59.882101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:01:59.882596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:00.026340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:00.027438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:00.027708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.028684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:00.028684 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.028684 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:00.028684 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.028684 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.028684 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:00.028684 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.028684 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:00.030318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.030959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:00.031727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:00.031727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:00.032236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:00.032678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.033569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:00.033569 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.033569 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:00.033569 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.033569 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.033569 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:00.033569 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.033569 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:00.034915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:00.035860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:00.035860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.035860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:00.035860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.035860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.035860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:00.035860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.035860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:00.036535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.037129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.037567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:00.037567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.037567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:00.037567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:00.037567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:00.038027 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:00.038194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.039989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:00.039989 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.039989 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:00.039989 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.039989 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.039989 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:00.039989 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.039989 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:00.041257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:00.042131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:00.042131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.042131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:00.042131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.042131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.042131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:00.042131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.042131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:00.042886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.043289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:00.043772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.043772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:00.126708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:00.126708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:00.127209 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:00.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:00.127500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.128205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:00.128205 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.128205 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:00.128205 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.128205 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.128205 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:00.128205 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.128205 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.129692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:00.130412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:04.846650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:04.848150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:04.879208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:04.879570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:04.880343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:04.880343 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:04.880343 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:04.880343 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:04.880343 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:04.880343 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:04.880343 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:04.880343 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:04.881125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:04.881687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:05.027076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:05.028000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:05.028379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.029358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:05.029358 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.029358 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:05.029358 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.029358 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.029358 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:05.029358 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.029358 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.031437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.032344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:05.033072 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:05.033072 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:05.033072 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:05.033072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.033619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:05.033619 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.033619 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:05.033619 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.033619 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.033619 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:05.033619 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.033619 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:05.034851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.035283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:05.036734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.037201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.038284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:05.038284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.038284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:05.038284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:05.038284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:05.038284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:05.038698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.038841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:05.038841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.038841 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:05.038841 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.038841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.038841 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:05.038841 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.038841 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:05.040102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:05.040574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:05.040574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.040574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:05.040574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.040574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.040574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:05.040574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.040574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:05.041768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.042244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:05.042728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.042728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:05.126698 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:05.127124 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:05.127555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:05.127917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:05.128125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.128836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:05.128836 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.128836 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:05.128836 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.128836 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.128836 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:05.128836 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.128836 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.130370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:05.131137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:09.847431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:09.848496 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:09.871459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:09.871884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:09.872685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:09.872685 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:09.872685 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:09.872685 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:09.872685 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:09.872685 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:09.872685 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:09.872685 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:09.873590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:09.874404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:10.028070 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:10.028849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:10.029209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.030000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:10.030000 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.030000 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:10.030000 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.030000 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.030000 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:10.030000 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.030000 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.030872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.031665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:10.032459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:10.032526 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:10.032872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:10.033166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.033860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:10.033860 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.033860 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:10.033860 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.033860 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.033860 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:10.033860 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.033860 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:10.035033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.035634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:10.036547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.037064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.037601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:10.037601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.037601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:10.037831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:10.038088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:10.038335 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:10.038560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.038986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:10.038986 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.038986 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:10.038986 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.038986 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.038986 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:10.038986 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.038986 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:10.039681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:10.040178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:10.040178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.040178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:10.040178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.040178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.040178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:10.040178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.040178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:10.040721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.041219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:10.041789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.041789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:10.127383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:10.127570 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:10.127876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:10.128160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:10.128278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.128781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:10.128781 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.128781 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:10.128781 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.128781 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.128781 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:10.128781 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.128781 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.129827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:10.130633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:14.848141 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:14.850573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:14.881714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:14.882118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:14.882928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:14.882928 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:14.882928 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:14.882928 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:14.882928 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:14.882928 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:14.882928 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:14.882928 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:14.884234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:14.885199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:15.028788 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:15.029803 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:15.030035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.030569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:15.030569 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.030569 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:15.030569 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.030569 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.030569 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:15.030569 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.030569 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.031654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.032446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:15.033253 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:15.033253 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:15.033556 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:15.033739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.034304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:15.034304 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.034304 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:15.034304 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.034304 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.034304 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:15.034304 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.034304 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:15.035706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.036455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:15.037327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.038073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.038921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:15.038921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.038921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:15.038921 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:15.038921 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:15.039424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:15.039853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.040860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:15.040860 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.040860 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:15.040860 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.040860 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.040860 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:15.040860 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.040860 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:15.042114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:15.042568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:15.042568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.042568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:15.042568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.042568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.042568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:15.042568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.042568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:15.043795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.044264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:15.044742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.044742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:15.127757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:15.128168 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:15.128796 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:15.129324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:15.129580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.130596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:15.130596 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.130596 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:15.130596 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.130596 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.130596 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:15.130596 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.130596 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.132227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:15.132958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:19.848913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:19.850526 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:19.852250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=256 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:02:19.887702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:19.888158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:19.889065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:19.889065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:19.889065 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:19.889065 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:19.889065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:19.889065 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:19.889065 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:19.889065 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:19.890406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:19.890852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:20.029623 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:20.030827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:20.031191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.032150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:20.032150 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.032150 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:20.032150 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.032150 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.032150 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:20.032150 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.032150 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.034195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.034969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:20.035763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:20.035763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:20.035763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:20.036043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.036853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:20.036853 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.036853 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:20.036853 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.036853 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.036853 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:20.036853 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.036853 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:20.038349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.038956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:20.041478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.042165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.042997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:20.042997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.042997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:20.042997 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:20.042997 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:20.042997 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:20.042997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.043319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:20.043319 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.043319 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:20.043319 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.043319 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.043319 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:20.043319 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.043319 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:20.044275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:20.044929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:20.044929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.044929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:20.044929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.044929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.044929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:20.044929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.044929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:20.046143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.046550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:20.047030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.047030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:20.128345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:20.128714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:20.129539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:20.129935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:20.130162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.130903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:20.130903 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.130903 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:20.130903 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.130903 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.130903 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:20.130903 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.130903 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.132086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:20.132905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:24.849459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:24.851075 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:24.892404 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:24.892561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:24.893050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:24.893050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:24.893050 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:24.893050 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:24.893050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:24.893050 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:24.893050 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:24.893050 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:24.893860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:24.894372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:25.030183 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:25.031489 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:25.031838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.032925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:25.032925 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.032925 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:25.032925 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.032925 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.032925 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:25.032925 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.032925 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.035037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.035791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:25.036586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:25.036586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:25.036586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:25.036903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.037862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:25.037862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.037862 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:25.037862 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.037862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.037862 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:25.037862 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.037862 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:25.039265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:25.040213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:25.040213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.040213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:25.040213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.040213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.040213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:25.040213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.040213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:25.040710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.041485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.042414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:25.042414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.042414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:25.042414 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:25.042648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:25.043162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:25.043500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.044399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:25.044399 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.044399 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:25.044399 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.044399 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.044399 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:25.044399 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.044399 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:25.045805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:25.046459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:25.046459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.046459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:25.046459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.046459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.046459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:25.046459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.046459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:25.047724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.048328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:25.048805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.048805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:25.129103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:25.129334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:25.129791 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:25.130068 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:25.130198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.130755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:25.130755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.130755 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:25.130755 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.130755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.130755 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:25.130755 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.130755 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.132052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:25.133056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:29.850158 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:29.851899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:29.890804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:29.891546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:29.891740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:29.891740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:29.891740 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:29.891740 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:29.891740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:29.891740 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:29.891740 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:29.891740 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:29.892548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:29.893258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:30.030946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:30.032190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:30.032542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.033512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:30.033512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.033512 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:30.033512 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.033512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.033512 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:30.033512 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.033512 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.035484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.036229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:30.036964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:30.036964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:30.036964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:30.037200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.037882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:30.037882 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.037882 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:30.037882 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.037882 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.037882 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:30.037882 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.037882 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:30.039585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.040345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:30.041463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.041880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.042349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:30.042349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.042349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:30.042602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:30.042925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:30.043481 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:30.043841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.044525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:30.044525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.044525 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:30.044525 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.044525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.044525 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:30.044525 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.044525 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:30.045646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:30.046260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:30.046260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.046260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:30.046260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.046260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.046260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:30.046260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.046260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:30.046970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.047462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:30.047939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.047939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:30.129639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:30.130164 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:30.130757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:30.131093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:30.131303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.132099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:30.132099 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.132099 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:30.132099 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.132099 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.132099 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:30.132099 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.132099 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.134336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:30.135192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:34.850876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:34.852523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:34.882119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:34.882387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:34.883121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:34.883121 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:34.883121 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:34.883121 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:34.883121 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:34.883121 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:34.883121 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:34.883121 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:34.883904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:34.884406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:35.031620 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:35.032772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:35.033223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.034186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:35.034186 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.034186 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:35.034186 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.034186 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.034186 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:35.034186 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.034186 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.035970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.036567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:35.037109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:35.037109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:35.037522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:35.037817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.038564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:35.038564 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.038564 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:35.038564 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.038564 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.038564 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:35.038564 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.038564 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:35.039839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.040286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:35.041570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.042086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.042590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:35.042590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.042590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:35.043065 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:35.043498 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:35.043990 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:35.044354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.045122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:35.045122 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.045122 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:35.045122 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.045122 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.045122 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:35.045122 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.045122 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:35.046343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:35.046853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:35.046853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.046853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:35.046853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.046853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.046853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:35.046853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.046853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:35.047969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.048470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:35.049113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.049113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:35.130434 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:35.131603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:35.132474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:35.132966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:35.133202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.135629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:35.136258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:35.136789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:35.136789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:35.136789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:35.136789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:39.851701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:39.855219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:39.887367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:39.887656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:39.889628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:39.890279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:39.890829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:39.890829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:39.890829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:39.890829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:39.890829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:39.890829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:40.032334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:40.034006 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:40.034390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.035365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:40.035365 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.035365 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:40.035365 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.035365 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.035365 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:40.035365 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.035365 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.037155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.038035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:40.038896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:40.038896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:40.038896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:40.038896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.039716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:40.039716 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.039716 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:40.039716 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.039716 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.039716 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:40.039716 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.039716 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:40.040740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.041521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:40.041949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.042754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.043722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:40.043722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.043722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:40.043722 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:40.044021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:40.044456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:40.044692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.045217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:40.045217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.045217 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:40.045217 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.045217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.045217 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:40.045217 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.045217 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:40.046503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:40.047024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:40.047024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.047024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:40.047024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.047024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.047024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:40.047024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.047024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:40.048122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.048585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:40.049104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.049104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:40.131474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:40.131877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:40.132761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:40.133244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:40.133470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.134403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:40.134403 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.134403 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:40.134403 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.134403 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.134403 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:40.134403 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.134403 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.136217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:40.136909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:44.853242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:44.854812 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:44.905220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:44.905508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:44.906298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:02:44.906298 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:44.906298 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:44.906298 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:44.906298 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:44.906298 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:44.906298 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:44.906298 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:44.907562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:44.908249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:45.033053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:45.033875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:45.034116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.034640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:45.034640 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.034640 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:45.034640 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.034640 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.034640 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:45.034640 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.034640 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.035731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.036185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:45.038094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:45.038678 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:45.039279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:45.039616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.040688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:45.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:45.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:45.040688 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.040688 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:45.042242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.042635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:45.043906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.044356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.044848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:45.044848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.044848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:45.044848 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:45.044848 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:45.045181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:45.045426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.046028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:45.046028 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.046028 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:45.046028 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.046028 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.046028 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:45.046028 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.046028 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:45.046752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.047124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:45.047829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.048245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:45.048705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.048705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:45.132089 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:45.132684 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:45.133289 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:45.133668 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:45.133858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.135813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:45.136395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:45.136997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:45.136997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:45.136997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:45.136997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:49.853772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:49.855352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:49.886998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:49.887140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:49.887710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:02:49.887710 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:49.887710 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:49.887710 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:49.887710 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:49.887710 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:49.887710 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:49.887710 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:49.888947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:49.889290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:50.034227 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:50.034769 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.035964 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.037084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.038173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:50.039039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:50.039039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:50.039406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.040145 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:50.040958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.041405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:50.042023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.042496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.043038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:50.043038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.043038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:50.044499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:50.044764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:50.045115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:50.045385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.045887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:50.045887 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.045887 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:50.045887 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.045887 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.045887 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:50.045887 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.045887 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:50.046726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:50.047490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:50.047490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.047490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:50.047490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.047490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.047490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:50.047490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.047490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:50.047967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.048635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:50.049420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.049420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:50.133484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:50.133877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:50.134870 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:50.135804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:50.135804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.136236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:50.136236 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.136236 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:50.136236 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.136236 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.136236 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:50.136236 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.136236 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.137738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:50.138219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:53.364383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=c8440d11-f2d0-48e3-9a23-05eefeafeda7, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6'}, gateway_chassis=[7c965bef-52b0-4e22-8cd4-38292bb33278], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:4e:d4:4a', name='lrp-14a0aa42-40aa-4508-9af3-cffdffef9818', networks=['172.24.5.142/24', '2001:db8::16a/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=105681) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 26 01:02:54.855026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:54.856677 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:54.894248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:54.894632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:54.895537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:02:54.895537 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:54.895537 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:54.895537 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:54.895537 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:54.895537 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:54.895537 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:54.895537 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:54.896453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:54.897387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:55.034411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:55.035833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:55.036200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:55.036492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.037352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:55.037352 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.037352 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:55.037352 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.037352 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.037352 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:55.037352 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.037352 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.039330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.040662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:55.041493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:55.041493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:02:55.041493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:55.041493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.041873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:55.041873 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.041873 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:55.041873 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.041873 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.041873 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:55.041873 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.041873 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:55.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:55.044325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:55.044325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.044325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:55.044325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.044325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.044325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:55.044325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.044325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:02:55.045348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.046062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.046592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:55.046592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.046592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:55.046592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:55.046592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:55.046592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:55.046773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.047448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:55.047448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.047448 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:55.047448 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.047448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.047448 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:55.047448 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.047448 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:02:55.048350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:55.049041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:02:55.049041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.049041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:02:55.049041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.049041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.049041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:02:55.049041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.049041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:02:55.049769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.050291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:55.051005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.051005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:55.134187 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:55.134591 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:55.135256 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:55.135609 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:55.135833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.136814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:55.136814 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.136814 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:55.136814 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.136814 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.136814 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:55.136814 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.136814 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.138431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:55.139150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:02:56.411300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=b8361734-94a3-4908-b9c8-acda0d473001, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'neutron:subnet_ids': '329ea1d3-f656-45b1-bf23-1d927d4578ea'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:9b:be:b9', name='lrp-31026846-35eb-400d-b822-d3eccb13b2a8', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=105681) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Mar 26 01:02:56.446114 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec synchronized; highest priority chassis 01b73fe7-f890-48b6-af16-36032e15d1ef
Mar 26 01:02:56.446846 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'f481c2f9-5052-46d5-821a-6c8712ec70ec', 'neutron:router_id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9'}}) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:56.447265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, chassis=01b73fe7-f890-48b6-af16-36032e15d1ef, priority=1, columns={}) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:56.448245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:56.448358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:02:56.448358 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:56.448358 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:56.448358 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:56.448358 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:56.448358 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:56.448358 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:56.448358 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event Traceback (most recent call last):
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     match.run(event, row, updates)
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2385, in link_network_ha_chassis_group
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     next(self.gen)
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     next(self.gen)
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     with self.create_transaction(
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     self.result = self.commit()
Mar 26 01:02:56.449484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     raise result.ex
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:56.450918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.event 
Mar 26 01:02:59.855756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:02:59.856939 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:02:59.889538 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:02:59.889994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:59.890909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:02:59.890909 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:59.890909 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:02:59.890909 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:02:59.890909 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:59.890909 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:02:59.890909 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:59.890909 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:59.891837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:02:59.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:00.035113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:00.036269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:00.036467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:00.036653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.037460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:00.037460 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.037460 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:00.037460 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.037460 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.037460 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:00.037460 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.037460 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:00.039356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.040088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:00.040891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:00.040891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:00.041272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:00.041575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.042361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:00.042361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.042361 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:00.042361 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.042361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.042361 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:00.042361 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.042361 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:00.043322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:00.044003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:00.044003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.044003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:00.044003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.044003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.044003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:00.044003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.044003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:00.044331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.044999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.045767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:00.045767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.045767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:00.046060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:00.046766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:00.047372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:00.047762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.048553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:00.048553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.048553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:00.048553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.048553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.048553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:00.048553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.048553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:00.049620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:00.050033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:00.050033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.050033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:00.050033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.050033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.050033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:00.050033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.050033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:00.050782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.051287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:00.051814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.051814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:00.134737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:00.135311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:00.135843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:00.136300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:00.136564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.137524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:00.137524 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.137524 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:00.137524 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.137524 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.137524 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:00.137524 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.137524 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.139203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:00.139932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:04.856448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:04.858207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:04.893573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:04.893819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:04.894445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:03:04.894445 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:04.894445 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:04.894445 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:04.894445 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:04.894445 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:04.894445 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:04.894445 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:04.895362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:04.895939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:05.035874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:05.037175 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:05.037518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:05.037756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.038740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:05.038740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.038740 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:05.038740 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.038740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.038740 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:05.038740 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.038740 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:05.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.041123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:05.041947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:05.041947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:05.041947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:05.041947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.042644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:05.042644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.042644 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:05.042644 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.042644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.042644 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:05.042644 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.042644 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:05.043784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:05.044574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:05.044574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.044574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:05.044574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.044574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.044574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:05.044574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.044574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:05.045172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.045876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.046721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:05.046721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.046721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:05.046721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:05.048680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:05.049224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:05.049450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.049905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:05.049905 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.049905 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:05.049905 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.049905 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.049905 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:05.049905 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.049905 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:05.050800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:05.051239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:05.051239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.051239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:05.051239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.051239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.051239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:05.051239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.051239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:05.051881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.052349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:05.052845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.052845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:05.135221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:05.135806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:05.136296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:05.136663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:05.136876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.137581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:05.137581 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.137581 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:05.137581 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.137581 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.137581 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:05.137581 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.137581 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.139196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:05.139547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:09.857188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:09.858313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:09.892205 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:09.892205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:09.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:03:09.892592 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:09.892592 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:09.892592 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:09.892592 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:09.892592 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:09.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:09.892592 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:09.893734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:09.894242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:10.036848 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:10.037852 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:10.038121 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:10.038283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.039024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:10.039024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.039024 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:10.039024 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.039024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.039024 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:10.039024 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.039024 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:10.040808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.041465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:10.042236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:10.042236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:10.042236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:10.042236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.043076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:10.043076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.043076 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:10.043076 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.043076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.043076 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:10.043076 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.043076 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:10.043966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:10.045330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:10.045330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.045330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:10.045330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.045330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.045330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:10.045330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.045330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:10.045768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.046401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.047138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:10.047138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.047138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:10.048560 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:10.048943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:10.049331 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:10.049540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.049990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:10.049990 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.049990 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:10.049990 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.049990 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.049990 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:10.049990 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.049990 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:10.050901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:10.051343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:10.051343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.051343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:10.051343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.051343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.051343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:10.051343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.051343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:10.051989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.052418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:10.052880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.052880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:10.135884 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:10.136574 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:10.137231 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:10.137651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:10.137951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.138712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:10.138712 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.138712 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:10.138712 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.138712 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.138712 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:10.138712 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.138712 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.139602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:10.140394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:14.857801 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:14.859282 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:14.968407 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:14.968827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:14.969897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:03:14.969897 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:14.969897 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:14.969897 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:14.969897 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:14.969897 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:14.969897 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:14.969897 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:14.971156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:14.971935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:15.037388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:15.039203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:15.039372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:15.039602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.040541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:15.040541 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.040541 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:15.040541 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.040541 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.040541 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:15.040541 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.040541 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:15.042220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.043131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.044180 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:15.045285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:15.045804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:15.045804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.045804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:15.045804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.045804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.045804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:15.045804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.045804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:15.046951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.047437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.048119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:15.048119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.048119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:15.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:15.049188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:15.049524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:15.049744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.050269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:15.050269 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.050269 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:15.050269 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.050269 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.050269 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:15.050269 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.050269 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:15.051462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:15.051967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:15.051967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.051967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:15.051967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.051967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.051967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:15.051967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.051967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:15.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.053859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:15.054425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.054425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:15.136386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:15.137129 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:15.137721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:15.138123 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:15.138343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.139146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:15.139146 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.139146 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:15.139146 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.139146 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.139146 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:15.139146 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.139146 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.140967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:15.141789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:19.858484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:19.860099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:19.861817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=257 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:03:19.898924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:19.899531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:19.900456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:03:19.900456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:19.900456 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:19.900456 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:19.900456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:19.900456 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:19.900456 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:19.900456 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:19.901630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:19.902470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:20.038091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:20.039308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:20.039589 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:20.039809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.040688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:20.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:20.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.040688 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:20.040688 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.040688 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:20.042478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.043175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:20.044081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:20.044081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:20.044081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:20.044081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.044675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:20.044675 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.044675 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:20.044675 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.044675 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.044675 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:20.044675 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.044675 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:20.045518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:20.046191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:20.046191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.046191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:20.046191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.046191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.046191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:20.046191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.046191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:20.046555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.047293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.048150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:20.048150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.048150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:20.049208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:20.049436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:20.049766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:20.050047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.050484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:20.050484 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.050484 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:20.050484 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.050484 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.050484 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:20.050484 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.050484 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:20.051237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:20.051715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:20.051715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.051715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:20.051715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.051715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.051715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:20.051715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.051715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:20.052178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.052634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:20.053185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.053185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:20.137197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:20.137613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:20.138197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:20.138465 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:20.138632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.139321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:20.139321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.139321 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:20.139321 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.139321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.139321 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:20.139321 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.139321 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.141154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:20.142247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:24.803466 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:24.812592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 26 01:03:24.812891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Number of inconsistencies found at create/update: networks=3, security_groups=1 {{(pid=105681) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}}
Mar 26 01:03:24.813072 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:03:24.846843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, resource={'id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'name': 'private', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['02d96a4b-4c93-4b95-ad0f-f11d085f0699', '0503b55a-1b69-49b5-b418-16ba37be90e9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T00:58:10Z', 'updated_at': '2026-03-26T00:58:13Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4449}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.847143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.847400 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6fe31530-ff9f-476b-aea1-4804059b76cd, external_ids_update=None, columns={'tag': 4449}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.847480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.848117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:03:24.848117 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:24.848117 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:24.848117 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:24.848117 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:24.848117 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:24.848117 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.848117 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:03:24.849190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:03:24.849819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:24.850666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:03:24.850666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.850666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:03:24.850666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:03:24.859608 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:24.863651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:24.902374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-db438906-c74f-4dc8-883c-6e56962e551b, resource={'id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'name': 'public', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['1c3e4583-6e63-4953-8050-3341662a5ec6', '23682989-62bf-4e07-a74f-0cb01077d5ac'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-26T00:58:16Z', 'updated_at': '2026-03-26T00:58:25Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.902746 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-db438906-c74f-4dc8-883c-6e56962e551b, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.903233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-200618ec-0540-41b0-9429-dc7dfc74abb0, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.903490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.907201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:03:24.907201 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:24.907201 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:24.907201 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:24.907201 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:24.907201 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:24.907201 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.907201 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:03:24.910709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:03:24.911042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:24.911419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:03:24.911419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.911419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:03:24.911858 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:03:24.924399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.925684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.928760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:03:24.928760 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:24.928760 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:24.928760 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:24.928760 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:24.928760 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:24.928760 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.928760 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:24.932948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.933805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:24.961951 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, resource={'id': 'f7e819e4-7101-412b-b047-baa5e16b2a52', 'name': 'admin_net', 'tenant_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'project_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f2e900ee-4d88-4151-8e50-661745950632'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:14Z', 'updated_at': '2026-03-26T01:02:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4803}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.962161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.962338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d43d91f1-f24a-411f-bae7-af85048f95b9, external_ids_update=None, columns={'tag': 4803}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:24.962423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.962955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:03:24.962955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:24.962955 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:24.962955 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:24.962955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:24.962955 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:24.962955 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.962955 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:03:24.963890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:03:24.964455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:24.966036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:03:24.966036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:24.966036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:03:24.966036 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource 6bd0245f-6dfe-46bd-92d9-7a152599c86b (type: security_groups) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:03:25.038487 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:25.039474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:25.039651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:25.039764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.040439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:25.040439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.040439 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:25.040439 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.040439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.040439 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:25.040439 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.040439 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:25.042217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.042690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:25.043261 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:25.043635 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:25.044061 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:25.044304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.044870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:25.044870 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.044870 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:25.044870 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.044870 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.044870 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:25.044870 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.044870 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:25.045775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:25.046277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:25.046277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.046277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:25.046277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.046277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.046277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:25.046277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.046277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:25.046855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.047322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.047823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:25.047823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.047823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:25.049528 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:25.049936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:25.050315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:25.050630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.051235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:25.051235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.051235 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:25.051235 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.051235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.051235 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:25.051235 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.051235 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:25.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:25.052752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:25.052752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.052752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:25.052752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.052752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.052752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:25.052752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.052752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:25.053533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.054023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:25.054527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.054527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:25.137668 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:25.138107 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:25.138774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:25.139367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:25.139509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.143730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:25.143730 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.143730 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:25.143730 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.143730 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.143730 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:25.143730 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.143730 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.144366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:25.144855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:25.739391 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.db.ovn_revision_numbers_db [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Successfully bumped revision number for resource 6bd0245f-6dfe-46bd-92d9-7a152599c86b (type: security_groups) to 9
Mar 26 01:03:25.739555 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Synchronization completed (took 0.93 seconds)
Mar 26 01:03:29.861447 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:29.863009 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:29.901928 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:29.902367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:29.903340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:03:29.903340 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:29.903340 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:29.903340 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:29.903340 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:29.903340 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:29.903340 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:29.903340 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:29.904301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:29.905083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:30.039094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:30.041360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:30.041684 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:30.041946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.042998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:30.042998 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.042998 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:30.042998 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.042998 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.042998 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:30.042998 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.042998 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:30.045349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.067796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.068597 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:30.069117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.069750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:30.104602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:30.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.106220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.107335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.108828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:30.109677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.110792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.110792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:30.110792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.110792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:30.138339 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:30.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:30.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:30.139641 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:30.139792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.140558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:30.140558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.140558 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:30.140558 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.140558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.140558 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:30.140558 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.140558 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.142091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:30.142572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:34.862334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:34.863598 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:34.894861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:34.895106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:34.895644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:03:34.895644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:34.895644 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:34.895644 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:34.895644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:34.895644 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:34.895644 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:34.895644 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:34.896706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:34.897468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:35.040531 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:35.041798 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:35.042152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:35.042374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.043219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:35.043219 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.043219 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:35.043219 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.043219 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.043219 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:35.043219 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.043219 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:35.044697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.214109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.215374 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:35.215836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.216418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:35.216935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:35.217644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.218095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.218695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.219256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:35.219788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:35.220347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.221647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:35.223001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:35.223719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:35.223719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:35.223719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:35.223719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:39.868438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:39.868817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:39.902545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:39.902792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:39.903444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:03:39.903444 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:39.903444 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:39.903444 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:39.903444 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:39.903444 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:39.903444 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:39.903444 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:39.904227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:39.904753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:40.041354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:40.042925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:40.043971 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:40.044307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.045508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:40.045508 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.045508 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:40.045508 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.045508 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.045508 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:40.045508 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.045508 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:40.047701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.048422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:40.049247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:40.049247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:40.049247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:40.049247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.050082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:40.050082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.050082 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:40.050082 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.050082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.050082 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:40.050082 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.050082 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:40.050840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:40.051555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:40.051555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.051555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:40.051555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.051555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.051555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:40.051555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.051555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:40.051889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.052332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.052817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:40.052817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.052817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:40.052817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:40.052817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:40.053069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:40.053325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.053751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:40.053751 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.053751 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:40.053751 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.053751 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.053751 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:40.053751 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.053751 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:40.054777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:40.055207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:40.055207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.055207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:40.055207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.055207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.055207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:40.055207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.055207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:40.055749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.056225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:40.056680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.056680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:40.139351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:40.139948 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:40.140528 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:40.140855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:40.141082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.141877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:40.141877 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.141877 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:40.141877 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.141877 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.141877 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:40.141877 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.141877 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.143609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:40.144563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:44.869210 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:44.871191 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:44.906761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:44.906949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:44.908064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:03:44.908064 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:44.908064 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:44.908064 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:44.908064 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:44.908064 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:44.908064 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:44.908064 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:44.909214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:44.910397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:45.042346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:45.045069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:45.045069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:45.045069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.045328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:45.045328 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.045328 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:45.045328 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.045328 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.045328 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:45.045328 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.045328 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:45.047137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.048080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:45.048973 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:45.048973 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:45.050404 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:45.050404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.050556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:45.050556 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.050556 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:45.050556 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.050556 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.050556 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:45.050556 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.050556 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:45.051838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:45.052606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:45.052606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.052606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:45.052606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.052606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.052606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:45.052606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.052606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:45.053382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.053922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.054474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:45.054474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.054474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:45.054474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:45.054474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:45.054875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:45.055213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.055831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:45.055831 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.055831 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:45.055831 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.055831 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.055831 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:45.055831 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.055831 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:45.057319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:45.057774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:45.057774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.057774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:45.057774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.057774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.057774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:45.057774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.057774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:45.058921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.059368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:45.059894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.059894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:45.139845 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:45.140186 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:45.142082 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:45.142082 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:45.142082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.142292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:45.142292 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.142292 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:45.142292 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.142292 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.142292 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:45.142292 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.142292 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.143659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:45.144478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:49.870007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:49.871844 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:49.904504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:49.904918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:49.905817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:03:49.905817 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:49.905817 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:49.905817 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:49.905817 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:49.905817 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:49.905817 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:49.905817 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:49.907320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:49.908141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:50.043245 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:50.044470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:50.044817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:50.045105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.046082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:50.046082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.046082 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:50.046082 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.046082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.046082 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:50.046082 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.046082 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:50.047632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.048367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:50.049184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:50.049274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:50.049719 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:50.050002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.050768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:50.050768 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.050768 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:50.050768 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.050768 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.050768 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:50.050768 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.050768 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:50.051488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:50.052161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:50.052161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.052161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:50.052161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.052161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.052161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:50.052161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.052161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:50.052509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.053200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.054020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:50.054020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.054020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:50.054295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:50.055170 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:50.055356 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:50.055697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.056691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:50.056691 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.056691 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:50.056691 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.056691 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.056691 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:50.056691 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.056691 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:50.057191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:50.057928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:50.057928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.057928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:50.057928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.057928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.057928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:50.057928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.057928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:50.058550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.059449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:50.060697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.060697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:50.140602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:50.141053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:50.141477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:50.141724 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:50.141870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.142436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:50.142436 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.142436 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:50.142436 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.142436 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.142436 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:50.142436 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.142436 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.144189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:50.145091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:54.870688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:54.872256 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:54.907903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:54.908407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:54.909381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:03:54.909381 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:54.909381 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:54.909381 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:54.909381 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:54.909381 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:54.909381 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:54.909381 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:54.910258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:54.910774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:55.043994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:55.045213 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:55.045515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:55.045748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.046628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:03:55.046628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.046628 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:55.046628 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.046628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.046628 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:55.046628 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.046628 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:55.048276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.048773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:55.049407 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:55.049795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:03:55.050269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:55.050532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.051233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:55.051233 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.051233 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:55.051233 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.051233 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.051233 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:55.051233 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.051233 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:55.052489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:55.053255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:55.053255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.053255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:55.053255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.053255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.053255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:55.053255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.053255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:03:55.053718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.055189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:55.055189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.055189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:55.055189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:55.055189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:55.055526 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:55.055860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.056547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:55.056547 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.056547 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:55.056547 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.056547 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.056547 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:55.056547 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.056547 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:03:55.057676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:55.058130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:03:55.058130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.058130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:03:55.058130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.058130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.058130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:03:55.058130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.058130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:03:55.059438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.059925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:55.060451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.060451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:55.141160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:55.141647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:55.142329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:55.142699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:55.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.143735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:03:55.143735 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.143735 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:55.143735 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.143735 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.143735 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:55.143735 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.143735 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.145708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:55.146249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:03:59.872423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:03:59.873270 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:03:59.909405 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:03:59.909686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:59.910365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:03:59.910365 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:59.910365 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:03:59.910365 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:03:59.910365 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:59.910365 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:03:59.910365 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:59.910365 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:59.911612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:03:59.913281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:00.044709 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:00.046060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:00.046312 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:00.046446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:00.048844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:00.049376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.049900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:00.049900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.049900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.049900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:00.049900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.049900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:00.050333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:00.050942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:00.051449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:00.051722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.052532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:00.052532 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.052532 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:00.052532 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.052532 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.052532 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:00.052532 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.052532 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:00.054103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:00.054590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:00.054590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.054590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:00.054590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.054590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.054590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:00.054590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.054590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:00.055764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.056752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.057252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:00.057252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.057252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:00.057252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:00.057252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:00.057252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:00.057252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.057577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:00.057577 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.057577 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:00.057577 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.057577 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.057577 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:00.057577 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.057577 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:00.059948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.060578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:00.061222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.061744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:00.141744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:00.142253 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:00.142878 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:00.143131 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:00.143279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.143843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:00.143843 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.143843 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:00.143843 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.143843 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.143843 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:00.143843 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.143843 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.145646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:00.146664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:04.872354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:04.873781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:04.911513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:04.911823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:04.912631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:04.912631 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:04.912631 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:04.912631 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:04.912631 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:04.912631 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:04.912631 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:04.912631 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:04.913805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:04.915012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:05.045589 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:05.046943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:05.047320 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:05.047485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:05.050144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:05.050925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.051661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:05.051661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.051661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.051661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:05.051661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.051661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:05.052769 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:05.053247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:05.053819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:05.054160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.054855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:05.054855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.054855 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:05.054855 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.054855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.054855 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:05.054855 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.054855 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:05.056568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:05.057043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:05.057043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.057043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:05.057043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.057043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.057043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:05.057043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.057043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:05.057486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:05.059678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.060090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.060411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:05.060411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.060411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:05.060411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:05.060840 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:05.061027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.061812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:05.061812 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.061812 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:05.061812 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.061812 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.061812 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:05.061812 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.061812 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:05.063145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:05.064042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:05.064042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.064042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:05.064042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.064042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.064042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:05.064042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.064042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:05.064758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.065319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:05.065808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.065808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:05.142285 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:05.142578 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:05.143091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:05.143429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:05.143643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.144356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:05.144356 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.144356 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:05.144356 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.144356 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.144356 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:05.144356 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.144356 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.146103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:05.146894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:09.872938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:09.874801 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:09.914371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:09.914757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:09.915214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:09.915214 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:09.915214 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:09.915214 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:09.915214 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:09.915214 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:09.915214 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:09.915214 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:09.916494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:09.917359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:10.046569 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:10.048166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:10.048502 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:10.048736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.049731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:10.049731 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.049731 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:10.049731 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.049731 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.049731 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:10.049731 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.049731 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:10.052190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.053236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:10.054605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:10.054605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:10.054605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:10.054605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.054936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:10.054936 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.054936 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:10.054936 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.054936 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.054936 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:10.054936 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.054936 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:10.056441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:10.057026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:10.057026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.057026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:10.057026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.057026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.057026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:10.057026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.057026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:10.058436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.058913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.059461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:10.059461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.059461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:10.059461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:10.059461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:10.059921 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:10.060294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.060967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:10.060967 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.060967 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:10.060967 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.060967 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.060967 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:10.060967 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.060967 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:10.061832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:10.062318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:10.062318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.062318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:10.062318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.062318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.062318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:10.062318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.062318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:10.062923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.063406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:10.063998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.063998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:10.143082 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:10.143558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:10.144223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:10.144588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:10.144800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.145550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:10.145550 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.145550 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:10.145550 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.145550 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.145550 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:10.145550 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.145550 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.146704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:10.147793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:14.873709 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:14.875571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:14.906760 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:14.907266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:14.908171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:14.908171 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:14.908171 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:14.908171 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:14.908171 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:14.908171 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:14.908171 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:14.908171 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:14.909278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:14.910099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:15.047277 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:15.048564 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:15.048897 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:15.049142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.050261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:15.050261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.050261 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:15.050261 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.050261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.050261 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:15.050261 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.050261 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:15.051932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.053304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:15.054064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:15.054064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:15.054911 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:15.054911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.055608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:15.055608 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.055608 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:15.055608 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.055608 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.055608 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:15.055608 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.055608 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:15.056839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.058517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:15.060135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:15.061462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.062468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.063404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.063748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:15.064055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.064384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.064384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:15.064384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.064384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:15.143772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:15.144176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:15.144686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:15.145065 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:15.145290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.146589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:15.146589 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.146589 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:15.146589 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.146589 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.146589 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:15.146589 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.146589 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.147852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:15.148407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:19.874875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:19.876671 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:19.878480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=324,neutron:UPDATE=1 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:04:19.921026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:19.921390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:19.922238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:19.922238 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:19.922238 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:19.922238 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:19.922238 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:19.922238 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:19.922238 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:19.922238 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:19.923466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:19.924598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:20.048038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:20.049463 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:20.049802 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:20.050076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.051191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:20.051191 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.051191 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:20.051191 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.051191 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.051191 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:20.051191 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.051191 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:20.052908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.053628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:20.054434 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:20.054495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:20.055119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:20.055435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.056192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:20.056192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.056192 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:20.056192 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.056192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.056192 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:20.056192 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.056192 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:20.057447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:20.058134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:20.058134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.058134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:20.058134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.058134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.058134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:20.058134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.058134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:20.058947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.059484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.059988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:20.059988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.059988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:20.059988 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:20.060111 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:20.060571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:20.060928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.061728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:20.061728 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.061728 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:20.061728 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.061728 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.061728 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:20.061728 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.061728 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:20.062994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:20.063458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:20.063458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.063458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:20.063458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.063458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.063458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:20.063458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.063458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:20.064552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.065024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:20.065518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.065518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:20.144711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:20.145243 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:20.145814 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:20.146522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:20.146787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.147520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:20.147520 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.147520 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:20.147520 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.147520 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.147520 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:20.147520 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.147520 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.149260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:20.150087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:24.875825 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:24.877869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:24.922091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:24.922411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:24.923195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:24.923195 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:24.923195 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:24.923195 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:24.923195 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:24.923195 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:24.923195 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:24.923195 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:24.924117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:24.925238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:25.048908 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:25.050265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:25.050603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:25.050845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.051963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:25.051963 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.051963 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:25.051963 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.051963 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.051963 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:25.051963 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.051963 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:25.054039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.054530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:25.055067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:25.055067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:25.055568 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:25.055869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.057135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:25.057135 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.057135 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:25.057135 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.057135 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.057135 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:25.057135 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.057135 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:25.058396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:25.058858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:25.058858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.058858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:25.058858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.058858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.058858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:25.058858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.058858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:25.059394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.059810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.060261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:25.060261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.060261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:25.060261 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:25.060423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:25.060829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:25.061258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.062204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:25.062204 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.062204 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:25.062204 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.062204 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.062204 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:25.062204 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.062204 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:25.063477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:25.063799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:25.063799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.063799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:25.063799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.063799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.063799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:25.063799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.063799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:25.065457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.066034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:25.066543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.066543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:25.145197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:25.145751 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:25.146590 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:25.147087 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:25.147332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.148258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:25.148258 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.148258 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:25.148258 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.148258 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.148258 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:25.148258 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.148258 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.149960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:25.150521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:29.876650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:29.877945 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:29.909143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:29.909596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:29.910599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:29.910599 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:29.910599 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:29.910599 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:29.910599 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:29.910599 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:29.910599 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:29.910599 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:29.911674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:29.912434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:30.049593 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:30.050907 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:30.051231 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:30.051457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:30.052331 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.052331 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:30.052331 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.052331 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.052331 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:30.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.052331 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:30.053935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.054695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:30.055162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:30.055197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:30.055588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:30.055867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.056603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:30.056603 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.056603 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:30.056603 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.056603 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.056603 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:30.056603 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.056603 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:30.057661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:30.058099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:30.058099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.058099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:30.058099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.058099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.058099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:30.058099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.058099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:30.059133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.059561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.060023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:30.060023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.060023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:30.060772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:30.061346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:30.061666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:30.062036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:30.064698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.065186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:30.065550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.066007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:30.145675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:30.146335 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:30.146889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:30.147305 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:30.147523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.148321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:30.148321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.148321 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:30.148321 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.148321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.148321 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:30.148321 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.148321 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.149966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:30.152144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:34.877393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:34.878559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:34.910808 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:34.911229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:34.912042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:34.912042 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:34.912042 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:34.912042 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:34.912042 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:34.912042 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:34.912042 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:34.912042 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:34.913106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:34.913937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:35.050382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:35.051676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:35.052074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:35.052338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.053362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:35.053362 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.053362 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:35.053362 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.053362 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.053362 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:35.053362 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.053362 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:35.055091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.055819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:35.056672 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:35.056672 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:35.056672 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:35.056672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.057456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:35.057456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.057456 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:35.057456 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.057456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.057456 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:35.057456 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.057456 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:35.058542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:35.059056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:35.059056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.059056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:35.059056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.059056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.059056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:35.059056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.059056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:35.059990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.060460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.061030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:35.061030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.061030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:35.061280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:35.061599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:35.062085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:35.062446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.063100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:35.063100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.063100 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:35.063100 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.063100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.063100 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:35.063100 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.063100 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:35.063827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:35.064311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:35.064311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.064311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:35.064311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.064311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.064311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:35.064311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.064311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:35.064830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.065374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:35.065898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.065898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:35.146388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:35.146885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:35.147492 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:35.147905 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:35.148196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.149065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:35.149065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.149065 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:35.149065 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.149065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.149065 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:35.149065 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.149065 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.150736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:35.151558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:39.878265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:39.879753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:39.913878 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:39.914367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:39.915248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:39.915248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:39.915248 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:39.915248 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:39.915248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:39.915248 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:39.915248 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:39.915248 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:39.916245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:39.917705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:40.051195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:40.052440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:40.052764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:40.053050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.054021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:40.054021 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.054021 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:40.054021 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.054021 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.054021 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:40.054021 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.054021 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:40.056020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.056874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:40.058049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:40.058049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:40.058049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:40.058049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.058392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:40.058392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.058392 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:40.058392 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.058392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.058392 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:40.058392 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.058392 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:40.059676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:40.060556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:40.060556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.060556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:40.060556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.060556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.060556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:40.060556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.060556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:40.061002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.061761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.062742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:40.062742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.062742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:40.062742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:40.062742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:40.062742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:40.062742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.063431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:40.063431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.063431 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:40.063431 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.063431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.063431 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:40.063431 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.063431 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:40.065301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:40.065750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:40.065750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.065750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:40.065750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.065750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.065750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:40.065750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.065750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:40.066924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.067407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:40.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:40.147078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:40.147611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:40.148346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:40.148725 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:40.148999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.149784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:40.149784 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.149784 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:40.149784 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.149784 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.149784 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:40.149784 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.149784 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.151474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:40.152505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:44.879054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:44.881222 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:44.914794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:44.915103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:44.915763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:44.915763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:44.915763 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:44.915763 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:44.915763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:44.915763 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:44.915763 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:44.915763 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:44.917117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:44.917607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:45.051771 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:45.052893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:45.053248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:45.053471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.054327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:45.054327 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.054327 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:45.054327 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.054327 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.054327 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:45.054327 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.054327 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:45.055310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.055632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:45.056681 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:45.056901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:45.057345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:45.057629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.058253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:45.058253 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.058253 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:45.058253 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.058253 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.058253 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:45.058253 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.058253 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:45.059178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:45.059498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:45.059498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.059498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:45.059498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.059498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.059498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:45.059498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.059498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:45.060175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.060492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.060833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:45.060833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.060833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:45.061908 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:45.062140 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:45.062516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:45.062873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.063453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:45.063453 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.063453 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:45.063453 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.063453 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.063453 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:45.063453 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.063453 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:45.063963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:45.064282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:45.064282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.064282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:45.064282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.064282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.064282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:45.064282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.064282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:45.064680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.064992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:45.065351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.065351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:45.147711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:45.148130 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:45.148775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:45.149206 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:45.149433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.150165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:45.150165 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.150165 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:45.150165 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.150165 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.150165 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:45.150165 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.150165 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.151200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:45.151525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:49.879624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:49.881171 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:49.913249 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:49.913654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:49.914605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:49.914605 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:49.914605 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:49.914605 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:49.914605 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:49.914605 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:49.914605 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:49.914605 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:49.915473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:49.916299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:50.052636 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:50.053846 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:50.054184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:50.054415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.055414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:50.055414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.055414 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:50.055414 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.055414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.055414 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:50.055414 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.055414 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:50.056838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.057416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:50.057932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:50.057932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:50.057932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:50.058252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.058862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:50.058862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.058862 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:50.058862 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.058862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.058862 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:50.058862 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.058862 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:50.059717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:50.060337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:50.060337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.060337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:50.060337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.060337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.060337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:50.060337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.060337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:50.060644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.061121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.061607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:50.061607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.061607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:50.062199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:50.062723 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:50.063262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:50.063702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.064355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:50.064355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.064355 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:50.064355 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.064355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.064355 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:50.064355 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.064355 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:50.065826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:50.066869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:50.066869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.066869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:50.066869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.066869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.066869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:50.066869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.066869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:50.067246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.067945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:50.068714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.068714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:50.148270 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:50.148675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:50.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:50.149558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:50.149807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.150466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:50.150466 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.150466 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:50.150466 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.150466 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.150466 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:50.150466 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.150466 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.152184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:50.153124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:54.880461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:54.882140 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:54.918116 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:54.918444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:54.919164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:54.919164 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:54.919164 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:54.919164 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:54.919164 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:54.919164 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:54.919164 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:54.919164 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:54.919929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:54.920771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:55.053429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:55.055220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:55.055551 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:55.055810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.056830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:55.056830 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.056830 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:55.056830 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.056830 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.056830 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:55.056830 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.056830 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:55.059106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.060168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:55.061366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:55.061366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:04:55.061366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:55.061366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.061720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:04:55.061720 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.061720 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:55.061720 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.061720 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.061720 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:55.061720 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.061720 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:55.062565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.063296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:04:55.063996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.064610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.065127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:55.065127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.065127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:55.065127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:55.065127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:55.065127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:55.065127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.065889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:55.065889 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.065889 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:55.065889 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.065889 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.065889 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:55.065889 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.065889 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:04:55.066625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:55.067068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:04:55.067068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.067068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:04:55.067068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.067068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.067068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:04:55.067068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.067068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:04:55.067561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.068023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:55.068904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.068904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:55.148804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:55.149193 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:55.149859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:55.150296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:55.150611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.151565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:04:55.151565 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.151565 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:55.151565 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.151565 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.151565 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:55.151565 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.151565 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.153265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:55.154179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:04:59.881304 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:04:59.882887 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:04:59.916157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:04:59.916597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:59.917481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:04:59.917481 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:59.917481 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:04:59.917481 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:04:59.917481 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:59.917481 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:04:59.917481 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:59.917481 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:59.918356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:04:59.919146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:00.054097 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:00.055478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:00.055690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:00.055796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.056668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:05:00.056668 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.056668 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:00.056668 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.056668 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.056668 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:00.056668 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.056668 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:00.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.059347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:00.060370 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:00.060475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:00.060826 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:00.060987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.061458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:00.061458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.061458 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:00.061458 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.061458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.061458 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:00.061458 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.061458 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:00.062336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:00.063139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:00.063139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.063139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:00.063139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.063139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.063139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:00.063139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.063139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:00.063551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.064197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.064776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:00.064776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.064776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:00.064776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:00.064921 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:00.065355 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:00.065711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.066429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:00.066429 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.066429 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:00.066429 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.066429 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.066429 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:00.066429 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.066429 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:00.067214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:00.067716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:00.067716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.067716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:00.067716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.067716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.067716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:00.067716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.067716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:00.068290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.068779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:00.069362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.069362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:00.149415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:00.149817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:00.150351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:00.150620 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:00.150778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.151448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:00.151448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.151448 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:00.151448 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.151448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.151448 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:00.151448 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.151448 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.153154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:00.153880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:04.882122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:04.884239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:04.920184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:04.920525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:04.921398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:05:04.921398 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:04.921398 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:04.921398 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:04.921398 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:04.921398 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:04.921398 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:04.921398 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:04.922576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:04.923485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:05.054796 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:05.056588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:05.056994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:05.057308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.058490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:05:05.058490 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.058490 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:05.058490 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.058490 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.058490 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:05.058490 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.058490 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:05.060522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.061294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:05.062055 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:05.062335 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:05.063146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:05.063374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.063960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:05:05.063960 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.063960 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:05.063960 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.063960 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.063960 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:05.063960 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.063960 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:05.065019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.065447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:05.065712 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:05.066684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.067175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.067643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:05.067643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.067643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:05.067838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:05.068370 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:05.068600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.069268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:05:05.069268 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.069268 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:05.069268 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.069268 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.069268 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:05.069268 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.069268 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:05.070041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.070411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:05.071164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.071581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:05.072123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.072123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:05.149875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:05.150390 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:05.150939 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:05.151425 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:05.151688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.152452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:05.152452 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.152452 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:05.152452 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.152452 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.152452 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:05.152452 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.152452 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.154262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:05.155281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:09.883076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:09.884684 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:09.918688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:09.919066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:09.919891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:05:09.919891 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:09.919891 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:09.919891 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:09.919891 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:09.919891 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:09.919891 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:09.919891 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:09.921176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:09.921953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:10.055602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:10.056816 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:10.057128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:10.057288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.057955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:10.057955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.057955 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:10.057955 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.057955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.057955 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:10.057955 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.057955 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.059724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.060472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:10.061693 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:10.061998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:10.062328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:10.062529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.062964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:10.062964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.062964 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:10.062964 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.062964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.062964 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:10.062964 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.062964 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:10.063706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:10.065333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.066212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:10.067005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:10.067247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:10.067564 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:10.067784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.068326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:05:10.068326 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.068326 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:10.068326 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.068326 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.068326 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:10.068326 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.068326 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:10.069131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.069652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:10.070197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.070675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:10.071231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.071231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:10.150540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:10.150844 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:10.151521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:10.151895 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:10.152136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.152927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:10.152927 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.152927 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:10.152927 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.152927 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.152927 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:10.152927 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.152927 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.154610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:10.155377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:14.883828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:14.885416 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:14.923359 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:14.923627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:14.924326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:05:14.924326 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:14.924326 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:14.924326 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:14.924326 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:14.924326 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:14.924326 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:14.924326 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:14.925535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:14.926405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:15.056378 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:15.057831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:15.058300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:15.058554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.059525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:15.059525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.059525 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:15.059525 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.059525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.059525 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:15.059525 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.059525 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.061449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.062348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:15.063181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:15.063181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:15.063181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:15.063379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.063910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:15.063910 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.063910 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:15.063910 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.063910 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.063910 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:15.063910 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.063910 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:15.064652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:15.065119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:15.065119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.065119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:15.065119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.065119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.065119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:15.065119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.065119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:15.065569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.066032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.066533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:15.066533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.066533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:15.067355 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:15.067668 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:15.068142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:15.068474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.069154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:05:15.069154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.069154 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:15.069154 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.069154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.069154 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:15.069154 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.069154 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:15.069904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.070373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:15.070929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.071399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:15.071889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.071889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:15.151325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:15.151768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:15.152410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:15.152944 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:15.153248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.154166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:15.154166 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.154166 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:15.154166 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.154166 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.154166 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:15.154166 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.154166 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.155925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:15.156476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:19.884596 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:19.886411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:19.893565 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=241 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:05:19.923084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:19.923555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:19.924788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:05:19.924788 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:19.924788 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:19.924788 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:19.924788 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:19.924788 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:19.924788 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:19.924788 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:19.926110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:19.926780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:20.057277 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:20.058565 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:20.058938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:20.059225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.060150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:20.060150 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.060150 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:20.060150 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.060150 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.060150 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:20.060150 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.060150 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.061829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.062683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:20.063480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:20.063480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:20.063480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:20.063759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.064405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:20.064405 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.064405 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:20.064405 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.064405 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.064405 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:20.064405 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.064405 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:20.065575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:20.066251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:20.066251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.066251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:20.066251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.066251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.066251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:20.066251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.066251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:20.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.067493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.068066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:20.068066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.068066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:20.068066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:20.068066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:20.068353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:20.068567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.069039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:05:20.069039 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.069039 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:20.069039 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.069039 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.069039 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:20.069039 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.069039 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:20.069737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.070183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:20.070718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.071365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:20.072048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.072048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:20.152185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:20.153207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:20.153874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:20.154354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:20.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.155641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:20.155641 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.155641 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:20.155641 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.155641 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.155641 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:20.155641 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.155641 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.157226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:20.157723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:24.885425 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:24.886831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:24.929843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:24.930250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:24.931151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:05:24.931151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:24.931151 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:24.931151 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:24.931151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:24.931151 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:24.931151 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:24.931151 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:24.932197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:24.932563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:25.057790 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:25.059049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:25.059214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:25.059330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.060149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:25.060149 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.060149 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:25.060149 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.060149 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.060149 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:25.060149 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.060149 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.061515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.061918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:25.063369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:25.063369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:25.067486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:25.067689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.068189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:25.070175 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:25.071071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:25.072018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:25.072018 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.072018 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:25.072018 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.072018 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.072018 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:25.072018 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.072018 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:25.072598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:25.079085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Traceback (most recent call last):
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.079473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:25.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.080192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:25.080493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.080765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-96349d3b-db4a-4f3d-a961-74a742262afd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:25.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.082843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:25.083224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.083224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:25.152752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:25.153201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:25.154495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:25.154495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:25.154495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.154626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:25.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:25.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:25.154626 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.154626 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.155599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:25.156122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:29.886026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:29.887855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:29.923590 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:29.924048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:29.924988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:05:29.924988 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:29.924988 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:29.924988 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:29.924988 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:29.924988 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:29.924988 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:29.924988 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:29.926242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:29.927318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:30.058600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:30.060606 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:30.060995 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:30.061331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.062449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:30.062449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.062449 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:30.062449 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.062449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.062449 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:30.062449 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.062449 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.064937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.065786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:30.066516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:30.066516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:30.066813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:30.067166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.067783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:30.067783 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.067783 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:30.067783 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.067783 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.067783 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:30.067783 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.067783 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:30.069348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:30.069895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:30.069895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.069895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:30.069895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.069895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.069895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:30.069895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.069895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:30.071310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.072165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.073195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:30.073195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.073195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:30.073195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:30.073195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:30.073195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:30.073195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.073694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:30.073694 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.073694 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:30.073694 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.073694 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.073694 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:30.073694 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.073694 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:30.074320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:30.075056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:30.075056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.075056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:30.075056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.075056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.075056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:30.075056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.075056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:30.075818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.076606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:30.077475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.077475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:30.153641 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:30.154133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:30.154904 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:30.155487 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:30.155852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.156913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:30.156913 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.156913 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:30.156913 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.156913 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.156913 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:30.156913 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.156913 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.157997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:30.158798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:34.886731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:34.888818 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:34.921998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:34.922527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:34.923622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:05:34.923622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:34.923622 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:34.923622 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:34.923622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:34.923622 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:34.923622 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:34.923622 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:34.924693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:34.925595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:35.059203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:35.060756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:35.061158 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:35.061464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.062583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:35.062583 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.062583 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:35.062583 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.062583 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.062583 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:35.062583 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.062583 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.064639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.065487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:35.066410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:35.066410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:35.066410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:35.066605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.067233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:35.067233 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.067233 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:35.067233 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.067233 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.067233 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:35.067233 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.067233 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:35.067853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.068699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:35.069506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.070492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:35.071942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:35.072310 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:35.072755 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:35.073134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.073885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:35.073885 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.073885 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:35.073885 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.073885 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.073885 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:35.073885 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.073885 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:35.075133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:35.075590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:35.075590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.075590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:35.075590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.075590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.075590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:35.075590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.075590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:35.076914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.077401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:35.077939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.077939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:35.154288 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:35.155053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:35.156002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:35.156581 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:35.156916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.158180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:35.158180 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.158180 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:35.158180 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.158180 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.158180 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:35.158180 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.158180 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.160017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:35.160540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:39.887391 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:39.889129 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:39.916423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:39.916861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:39.917711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:05:39.917711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:39.917711 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:39.917711 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:39.917711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:39.917711 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:39.917711 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:39.917711 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:39.919158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:39.919933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:40.059831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:40.061512 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:40.062002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:40.062285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.063398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:40.063398 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.063398 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:40.063398 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.063398 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.063398 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:40.063398 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.063398 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.065422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.066329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:40.067291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:40.067291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:40.067291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:40.067291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.067715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:40.067715 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.067715 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:40.067715 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.067715 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.067715 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:40.067715 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.067715 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:40.068508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.070235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:40.071925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:40.073663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.074913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.075819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.077308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:40.078175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.078696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.078696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:40.078696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.078696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:40.154828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:40.155521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:40.156203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:40.156619 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:40.156929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.157763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:40.157763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.157763 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:40.157763 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.157763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.157763 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:40.157763 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.157763 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.159540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:40.160528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:44.892819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:44.892819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:44.924151 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:44.924588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:44.925512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:05:44.925512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:44.925512 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:44.925512 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:44.925512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:44.925512 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:44.925512 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:44.925512 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:44.926719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:44.928151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:45.062471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:45.062471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:45.062697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:45.062861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.063851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:45.063851 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.063851 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:45.063851 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.063851 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.063851 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:45.063851 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.063851 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.065434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.066868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.069534 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:45.070328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.071999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:45.072628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:45.073205 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.077360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:45.077680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.077680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:45.078038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.078329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:45.078646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.078646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:45.155483 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:45.156301 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:45.156651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:45.157021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:45.157256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.158375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:45.158375 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.158375 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:45.158375 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.158375 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.158375 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:45.158375 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.158375 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.160467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:45.162263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:49.888651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:49.890190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:49.922441 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:49.923014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:49.924029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:05:49.924029 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:49.924029 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:49.924029 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:49.924029 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:49.924029 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:49.924029 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:49.924029 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:49.925122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:49.925899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:50.061456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:50.063205 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:50.063598 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:50.063891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.064764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:50.064764 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.064764 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:50.064764 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.064764 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.064764 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:50.064764 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.064764 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.066496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.067080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:50.067557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:50.067729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:50.068096 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:50.068254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.068852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:50.068852 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.068852 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:50.068852 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.068852 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.068852 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:50.068852 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.068852 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:50.069865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:50.070397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:50.070397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.070397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:50.070397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.070397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.070397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:50.070397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.070397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:50.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.071753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.072233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:50.072233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.072233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:50.072999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:50.073555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:50.073993 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:50.074214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.074827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:50.074827 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.074827 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:50.074827 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.074827 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.074827 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:50.074827 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.074827 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:50.076090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:50.076518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:50.076518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.076518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:50.076518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.076518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.076518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:50.076518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.076518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:50.077690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.078177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:50.078636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.078636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:50.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:50.156614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:50.157373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:50.157772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:50.158033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.158839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:50.158839 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.158839 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:50.158839 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.158839 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.158839 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:50.158839 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.158839 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.160485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:50.161298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:54.889281 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:54.890811 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:54.929342 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:54.929718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:54.930606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:05:54.930606 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:54.930606 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:54.930606 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:54.930606 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:54.930606 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:54.930606 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:54.930606 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:54.931496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:54.932265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:55.062452 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:55.064067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:55.064426 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:55.064659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.065441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:55.065441 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.065441 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:55.065441 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.065441 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.065441 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:55.065441 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.065441 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.066552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.067381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:55.068319 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:55.068319 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:05:55.068547 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:55.068822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.069485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:55.069485 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.069485 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:55.069485 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.069485 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.069485 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:55.069485 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.069485 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:55.070244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:55.071052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:55.071052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.071052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:55.071052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.071052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.071052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:55.071052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.071052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:05:55.071467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.072312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.073199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:55.073199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.073199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:55.073350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:55.073778 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:55.074272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:55.074616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.075314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:55.075314 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.075314 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:55.075314 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.075314 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.075314 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:55.075314 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.075314 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:05:55.076399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:55.076837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:05:55.076837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.076837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:05:55.076837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.076837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.076837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:05:55.076837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.076837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:05:55.077423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.077881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:55.078400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.078400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:55.156765 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:55.157239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:55.157527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:55.157694 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:55.157804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.158211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:05:55.158211 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.158211 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:55.158211 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.158211 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.158211 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:55.158211 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.158211 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.159439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:55.160169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:05:59.890156 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:05:59.891396 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:05:59.919629 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:05:59.920100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:59.921019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:05:59.921019 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:59.921019 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:05:59.921019 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:05:59.921019 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:59.921019 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:05:59.921019 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:59.921019 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:59.922209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:05:59.922611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:00.063557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.067051 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.068029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.068920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.069925 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:00.070884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:00.071771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:00.071771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.071771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:00.071771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.071771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.071771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:00.071771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.071771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:00.072188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.072994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.073692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:00.073692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.073692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:00.073692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:00.074142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:00.074537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:00.074782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.075300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:00.075300 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.075300 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:00.075300 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.075300 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.075300 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:00.075300 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.075300 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:00.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:00.077045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:00.077045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.077045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:00.077045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.077045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.077045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:00.077045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.077045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:00.078267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.078598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:00.078950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.078950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:00.157399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.160307 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.160901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:00.161370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:04.890946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:04.891963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:04.918262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:04.918543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:04.919194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:04.919194 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:04.919194 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:04.919194 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:04.919194 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:04.919194 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:04.919194 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:04.919194 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:04.920036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:04.920838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:05.064174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:05.065063 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:05.065276 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:05.065433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.066002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:05.066002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.066002 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:05.066002 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.066002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.066002 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:05.066002 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.066002 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.067637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.068473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:05.069350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:05.069350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:05.069475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:05.069723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.070168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:05.070168 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.070168 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:05.070168 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.070168 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.070168 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:05.070168 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.070168 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:05.071431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:05.072156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:05.072156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.072156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:05.072156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.072156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.072156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:05.072156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.072156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:05.072946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.073773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.074323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:05.074323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.074323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:05.074323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:05.074461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:05.074718 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:05.074930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.075439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:05.075439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.075439 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:05.075439 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.075439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.075439 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:05.075439 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.075439 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:05.076590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:05.077046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:05.077046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.077046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:05.077046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.077046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.077046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:05.077046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.077046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:05.078163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.078625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:05.079173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.079173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:05.157884 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:05.158481 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:05.158862 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:05.159076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:05.159207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.159787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:05.159787 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.159787 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:05.159787 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.159787 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.159787 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:05.159787 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.159787 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.162071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:05.162550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:09.891667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:09.893401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:09.927259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:09.927259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:09.927448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:09.927448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:09.927448 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:09.927448 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:09.927448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:09.927448 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:09.927448 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:09.927448 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:09.928485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:09.929282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:10.064682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:10.066075 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:10.066303 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:10.066438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.067258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:10.067258 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.067258 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:10.067258 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.067258 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.067258 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:10.067258 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.067258 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.068926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.069733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:10.070494 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:10.070494 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:10.070494 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:10.070646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.071335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:10.071335 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.071335 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:10.071335 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.071335 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.071335 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:10.071335 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.071335 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:10.071987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:10.072652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:10.072652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.072652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:10.072652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.072652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.072652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:10.072652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.072652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:10.073038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.073674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.074476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:10.074476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.074476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:10.074476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:10.074901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:10.075375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:10.075602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.076192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:10.076192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.076192 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:10.076192 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.076192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.076192 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:10.076192 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.076192 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:10.077469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:10.077895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:10.077895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.077895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:10.077895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.077895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.077895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:10.077895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.077895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:10.079448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.079877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:10.080405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.080405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:10.158481 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:10.158917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:10.159515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:10.159896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:10.160249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.161061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:10.161061 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.161061 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:10.161061 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.161061 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.161061 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:10.161061 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.161061 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.162992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:10.163752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:14.892401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:14.894234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:14.924206 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:14.924618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:14.925530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:14.925530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:14.925530 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:14.925530 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:14.925530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:14.925530 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:14.925530 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:14.925530 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:14.926402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:14.927263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:15.065267 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:15.066433 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:15.066738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:15.067017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.067857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:15.067857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.067857 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:15.067857 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.067857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.067857 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:15.067857 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.067857 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.069569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.070384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:15.071101 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:15.071101 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:15.071101 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:15.071355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.072218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:15.072218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.072218 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:15.072218 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.072218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.072218 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:15.072218 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.072218 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:15.073710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:15.074503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:15.074503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.074503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:15.074503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.074503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.074503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:15.074503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.074503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:15.075677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.076406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.077159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:15.077159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.077159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:15.077159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:15.077159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:15.077159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:15.079137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:15.079536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:15.079536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.079536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:15.079536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.079536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.079536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:15.079536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.079536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:15.080098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.080550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:15.081032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.081032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:15.159016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:15.159392 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:15.159898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:15.160292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:15.160518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.161333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:15.161333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.161333 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:15.161333 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.161333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.161333 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:15.161333 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.161333 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.163088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:15.163803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:19.893341 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:19.895141 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:19.901211 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:06:19.930336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:19.930733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:19.931593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:19.931593 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:19.931593 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:19.931593 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:19.931593 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:19.931593 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:19.931593 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:19.931593 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:19.932846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:19.933590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:20.065885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:20.066910 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:20.067299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:20.067539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.068356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:20.068356 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.068356 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:20.068356 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.068356 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.068356 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:20.068356 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.068356 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.070081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.070547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:20.071140 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:20.071140 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:20.071274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:20.071531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.072267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:20.072267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.072267 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:20.072267 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.072267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.072267 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:20.072267 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.072267 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:20.073256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:20.073715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:20.073715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.073715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:20.073715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.073715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.073715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:20.073715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.073715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:20.074203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.074751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.075727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:20.075727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.075727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:20.076459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:20.077188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:20.077531 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:20.077736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.078226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:20.078226 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.078226 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:20.078226 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.078226 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.078226 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:20.078226 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.078226 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:20.079275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:20.079916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:20.079916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.079916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:20.079916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.079916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.079916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:20.079916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.079916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:20.080640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.081151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:20.081608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.081608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:20.159579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:20.159939 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:20.160495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:20.160874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:20.161171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.161944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:20.161944 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.161944 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:20.161944 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.161944 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.161944 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:20.161944 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.161944 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.163571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:20.164445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:24.893925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:24.895228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:24.927886 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:24.928171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:24.929076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:24.929076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:24.929076 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:24.929076 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:24.929076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:24.929076 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:24.929076 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:24.929076 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:24.930290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:24.931122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:25.066303 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:25.067523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:25.067879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:25.068003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.069104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:25.069104 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.069104 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:25.069104 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.069104 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.069104 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:25.069104 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.069104 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.070578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.071072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:25.071672 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:25.071672 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:25.071745 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:25.072013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.072732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:25.072732 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.072732 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:25.072732 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.072732 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.072732 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:25.072732 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.072732 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:25.073894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:25.074348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:25.074348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.074348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:25.074348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.074348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.074348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:25.074348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.074348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:25.075451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.076089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.076637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:25.076637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.076637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:25.076959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:25.077334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:25.077903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:25.078395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.079202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:25.079202 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.079202 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:25.079202 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.079202 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.079202 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:25.079202 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.079202 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:25.080403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:25.080916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:25.080916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.080916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:25.080916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.080916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.080916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:25.080916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.080916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:25.081946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.082422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:25.082918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.082918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:25.160062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:25.160596 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:25.161073 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:25.161539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:25.161813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.162669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:25.162669 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.162669 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:25.162669 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.162669 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.162669 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:25.162669 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.162669 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.164237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:25.165040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:29.894816 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:29.895783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:29.926478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:29.926684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:29.927274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:29.927274 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:29.927274 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:29.927274 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:29.927274 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:29.927274 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:29.927274 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:29.927274 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:29.928203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:29.929023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:30.067200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:30.068863 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:30.069235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:30.069475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.070396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:30.070396 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.070396 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:30.070396 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.070396 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.070396 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:30.070396 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.070396 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.071780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.072582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:30.073498 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:30.073498 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:30.073498 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:30.073498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.074026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:30.074026 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.074026 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:30.074026 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.074026 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.074026 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:30.074026 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.074026 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:30.075312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:30.076146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:30.076146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.076146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:30.076146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.076146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.076146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:30.076146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.076146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:30.076880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.078338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:30.078338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.078338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:30.078338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:30.078338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:30.078338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:30.078338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.078759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:30.078759 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.078759 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:30.078759 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.078759 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.078759 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:30.078759 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.078759 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:30.080204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:30.080618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:30.080618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.080618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:30.080618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.080618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.080618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:30.080618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.080618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:30.081728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.082193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:30.082647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.082647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:30.160734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:30.161123 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:30.161573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:30.161901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:30.162181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.162840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:30.162840 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.162840 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:30.162840 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.162840 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.162840 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:30.162840 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.162840 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.164617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:30.165120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:34.895557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:34.897480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:34.934099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:34.934536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:34.935507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:34.935507 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:34.935507 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:34.935507 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:34.935507 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:34.935507 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:34.935507 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:34.935507 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:34.936672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:34.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:35.067806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:35.069014 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:35.069358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:35.069645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.070869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:35.070869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.070869 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:35.070869 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.070869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.070869 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:35.070869 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.070869 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.072682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.073476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:35.074406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:35.074406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:35.074406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:35.074406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.074763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:35.074763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.074763 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:35.074763 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.074763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.074763 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:35.074763 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.074763 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:35.076224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:35.076666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:35.076666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.076666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:35.076666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.076666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.076666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:35.076666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.076666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:35.077947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.078385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.078873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:35.078873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.078873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:35.078873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:35.078873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:35.079089 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:35.079329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.079857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:35.079857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.079857 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:35.079857 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.079857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.079857 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:35.079857 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.079857 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:35.080663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:35.081128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:35.081128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.081128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:35.081128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.081128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.081128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:35.081128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.081128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:35.081590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.082025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:35.082520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.082520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:35.161266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:35.161738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:35.162377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:35.162715 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:35.162939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.163880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:06:35.163880 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.163880 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:35.163880 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.163880 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.163880 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:35.163880 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.163880 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.165880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:35.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:39.896379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:39.898098 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:39.932516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:39.932912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:39.933774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:39.933774 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:39.933774 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:39.933774 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:39.933774 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:39.933774 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:39.933774 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:39.933774 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:39.934906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:39.935747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:40.068447 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:40.069658 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:40.070024 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:40.070257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.071070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:40.071070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.071070 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:40.071070 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.071070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.071070 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:40.071070 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.071070 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.072730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.073575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:40.074396 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:40.074809 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:40.075393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:40.075743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.076445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:40.076445 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.076445 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:40.076445 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.076445 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.076445 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:40.076445 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.076445 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:40.077338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:40.077810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:40.077810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.077810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:40.077810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.077810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.077810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:40.077810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.077810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:40.078293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.078727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.079202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:40.079202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.079202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:40.079202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:40.079202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:40.079491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:40.079805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.080482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:40.080482 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.080482 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:40.080482 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.080482 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.080482 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:40.080482 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.080482 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:40.081315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:40.081697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:40.081697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.081697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:40.081697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.081697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.081697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:40.081697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.081697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:40.082303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.082728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:40.083325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.083325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:40.161918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:40.162507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:40.162987 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:40.163398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:40.163669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.164484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:06:40.164484 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.164484 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:40.164484 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.164484 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.164484 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:40.164484 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.164484 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.166077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:40.166826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:44.897094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:44.898743 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:44.934495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:44.934864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:44.935683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:44.935683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:44.935683 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:44.935683 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:44.935683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:44.935683 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:44.935683 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:44.935683 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:44.936559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:44.937643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:45.069103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:45.070369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:45.070630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:45.070841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.071620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:45.071620 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.071620 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:45.071620 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.071620 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.071620 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:45.071620 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.071620 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.073153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.073917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:45.074791 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:45.075274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:45.075962 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:45.076278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.077047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:45.077047 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.077047 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:45.077047 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.077047 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.077047 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:45.077047 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.077047 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:45.078264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:45.078717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:45.078717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.078717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:45.078717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.078717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.078717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:45.078717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.078717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:45.079765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.080218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.080712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:45.080712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.080712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:45.080712 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:45.080712 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:45.081229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:45.081587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.082161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:45.082161 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.082161 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:45.082161 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.082161 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.082161 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:45.082161 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.082161 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:45.083407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:45.083843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:45.083843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.083843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:45.083843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.083843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.083843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:45.083843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.083843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:45.084954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.085397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:45.085869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.085869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:45.162538 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:45.163137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:45.163764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:45.164161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:45.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.165211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:06:45.165211 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.165211 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:45.165211 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.165211 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.165211 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:45.165211 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.165211 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.167106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:45.167880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:49.897749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:49.899399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:49.933387 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:49.933656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:49.934341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:49.934341 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:49.934341 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:49.934341 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:49.934341 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:49.934341 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:49.934341 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:49.934341 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:49.935222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:49.936067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:50.069774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:50.070844 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:50.070938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:50.071075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.071565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:50.071565 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.071565 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:50.071565 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.071565 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.071565 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:50.071565 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.071565 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.072599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.073140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:50.075555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:50.075555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:50.075782 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:50.075953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.077077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:50.077077 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.077077 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:50.077077 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.077077 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.077077 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:50.077077 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.077077 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:50.077797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:50.078353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:50.078353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.078353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:50.078353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.078353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.078353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:50.078353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.078353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:50.079182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.079662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.080242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:50.080242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.080242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:50.080363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:50.080594 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:50.081215 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:50.081215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.081653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:50.081653 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.081653 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:50.081653 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.081653 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.081653 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:50.081653 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.081653 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:50.082434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:50.082904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:50.082904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.082904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:50.082904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.082904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.082904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:50.082904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.082904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:50.083430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.083902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:50.084423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.084423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:50.163173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:50.163963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:50.164536 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:50.164824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:50.165210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.165965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:06:50.165965 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.165965 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:50.165965 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.165965 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.165965 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:50.165965 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.165965 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.168019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:50.168778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:54.898304 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:54.900153 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:54.935840 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:54.936118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:54.936783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:54.936783 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:54.936783 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:54.936783 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:54.936783 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:54.936783 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:54.936783 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:54.936783 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:54.937817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:54.938270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:55.070366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:55.071087 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:55.071247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:55.071377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.071910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:55.071910 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.071910 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:55.071910 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.071910 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.071910 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:55.071910 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.071910 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.072939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.073455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:55.075732 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:55.076162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:06:55.076507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:55.076674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.077278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:55.077278 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.077278 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:55.077278 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.077278 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.077278 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:55.077278 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.077278 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:55.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:55.078606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:55.078606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.078606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:55.078606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.078606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.078606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:55.078606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.078606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:06:55.079120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.079572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.080079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:55.080079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.080079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:55.080749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:55.081243 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:55.081605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:55.081813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.082300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:06:55.082300 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.082300 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:55.082300 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.082300 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.082300 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:55.082300 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.082300 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:06:55.083163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:55.083643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:06:55.083643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.083643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:06:55.083643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.083643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.083643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:06:55.083643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.083643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:06:55.084204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.084650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:55.085183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.085183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:55.165028 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:55.165028 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:55.165260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:55.165499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:55.165726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.166527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:06:55.166527 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.166527 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:55.166527 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.166527 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.166527 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:55.166527 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.166527 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.167679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:55.168044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:06:59.899010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:06:59.901795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:06:59.938541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:06:59.938913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:59.939946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:06:59.939946 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:59.939946 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:06:59.939946 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:06:59.939946 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:59.939946 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:06:59.939946 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:59.939946 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:59.941149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:06:59.941758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:00.071065 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:00.072487 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:00.072761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:00.073026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.073872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:00.073872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.073872 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:00.073872 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.073872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.073872 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:00.073872 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.073872 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.074818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.075612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:00.076141 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:00.076362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:00.077084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:00.077084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.077553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:00.077553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.077553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:00.077553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.077553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.077553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:00.077553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.077553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:00.078155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.078911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:00.079829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.081073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:00.081559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:00.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:00.082242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:00.082484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.083059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:00.083059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.083059 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:00.083059 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.083059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.083059 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:00.083059 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.083059 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:00.084369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:00.085081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:00.085081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.085081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:00.085081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.085081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.085081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:00.085081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.085081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:00.085892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.086408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:00.087097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.087097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:00.164599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:00.165362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:00.166053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:00.166557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:00.166698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.167686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:00.167686 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.167686 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:00.167686 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.167686 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.167686 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:00.167686 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.167686 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.169426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:00.170275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:04.899529 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:04.904831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:04.943401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:04.943857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:04.945917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:04.946678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:04.947278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:04.947278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:04.947278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:04.947278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:04.947278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:04.947278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:05.071682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.074424 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.075866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.076703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:05.077482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:05.077482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:05.079347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:05.079347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.079588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:05.079588 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.079588 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:05.079588 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.079588 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.079588 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:05.079588 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.079588 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:05.080095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.080907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:05.081421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:05.081995 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.083377 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:05.084108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:05.084580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:05.084580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.084580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:05.084580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.084580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.084580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:05.084580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.084580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:05.085089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.085658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:05.086232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.086232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:05.165280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:05.165680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.167805 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.169464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:05.170356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:09.901062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:09.902028 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:09.934426 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:09.934544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:09.934971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:09.934971 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:09.934971 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:09.934971 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:09.934971 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:09.934971 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:09.934971 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:09.934971 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:09.935730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:09.936568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:10.072256 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:10.073460 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:10.073750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:10.074028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.074807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:10.074807 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.074807 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:10.074807 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.074807 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.074807 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:10.074807 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.074807 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.075824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.076570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:10.077258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:10.077422 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:10.077922 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:10.078245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.078909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:10.078909 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.078909 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:10.078909 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.078909 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.078909 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:10.078909 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.078909 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:10.079816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:10.080501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:10.080501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.080501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:10.080501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.080501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.080501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:10.080501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.080501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:10.080862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.081466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.082592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:10.082592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.082592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:10.082592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:10.082592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:10.082592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:10.082859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.083181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:10.083181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.083181 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:10.083181 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.083181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.083181 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:10.083181 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.083181 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:10.083992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:10.084621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:10.084621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.084621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:10.084621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.084621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.084621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:10.084621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.084621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:10.085042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.085704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:10.086386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.086386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:10.166055 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:10.166689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:10.167284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:10.167657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:10.167889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.168750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:10.168750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.168750 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:10.168750 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.168750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.168750 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:10.168750 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.168750 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.170577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:10.171367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:14.901802 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:14.903078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:14.938952 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:14.939162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:14.939695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:14.939695 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:14.939695 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:14.939695 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:14.939695 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:14.939695 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:14.939695 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:14.939695 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:14.940714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:14.941560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:15.072857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:15.073989 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:15.074411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:15.074678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.075483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:15.075483 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.075483 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:15.075483 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.075483 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.075483 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:15.075483 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.075483 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.077048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.077577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:15.078085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:15.078085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:15.078085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:15.078282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.078915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:15.078915 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.078915 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:15.078915 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.078915 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.078915 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:15.078915 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.078915 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:15.080039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:15.080548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:15.080548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.080548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:15.080548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.080548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.080548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:15.080548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.080548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:15.081486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.081936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.082483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:15.082483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.082483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:15.082483 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:15.082776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:15.083288 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:15.083613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.084255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:15.084255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.084255 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:15.084255 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.084255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.084255 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:15.084255 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.084255 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:15.085147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:15.085570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:15.085570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.085570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:15.085570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.085570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.085570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:15.085570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.085570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:15.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.086647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:15.087223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.087223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:15.166696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:15.167230 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:15.167772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:15.168200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:15.168419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.169469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:15.169469 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.169469 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:15.169469 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.169469 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.169469 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:15.169469 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.169469 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.171609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:15.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:19.902469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:19.904102 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:19.905999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=239 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:07:19.939876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:19.940346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:19.941281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:19.941281 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:19.941281 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:19.941281 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:19.941281 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:19.941281 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:19.941281 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:19.941281 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:19.942429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:19.943325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:20.073639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:20.074996 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:20.075805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:20.075805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.076672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:20.076672 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.076672 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:20.076672 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.076672 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.076672 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:20.076672 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.076672 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.078405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.079211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:20.079952 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:20.079952 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:20.079952 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:20.080120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.080737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:20.080737 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.080737 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:20.080737 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.080737 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.080737 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:20.080737 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.080737 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:20.082122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:20.082580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:20.082580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.082580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:20.082580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.082580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.082580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:20.082580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.082580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:20.083931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.084386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.084885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:20.084885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.084885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:20.084885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:20.084885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:20.085235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:20.085459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.086063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:20.086063 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.086063 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:20.086063 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.086063 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.086063 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:20.086063 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.086063 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:20.087440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:20.087999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:20.087999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.087999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:20.087999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.087999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.087999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:20.087999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.087999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:20.089245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.089690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:20.090233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.090233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:20.167272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:20.167885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:20.168552 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:20.168955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:20.169259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.170151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:20.170151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.170151 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:20.170151 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.170151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.170151 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:20.170151 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.170151 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.171694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:20.172550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:24.903099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:24.904643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:24.934789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:24.935306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:24.936514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:24.936514 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:24.936514 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:24.936514 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:24.936514 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:24.936514 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:24.936514 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:24.936514 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:24.937689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:24.938074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:25.074239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:25.075970 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:25.076309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:25.076535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.077369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:25.077369 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.077369 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:25.077369 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.077369 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.077369 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:25.077369 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.077369 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.079488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.081425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.083701 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:25.084410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:25.084832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:25.084832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.084832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:25.084832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.084832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.084832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:25.084832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.084832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:25.085780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.086208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.086703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:25.086703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.086703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:25.086703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:25.086703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:25.086954 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:25.087328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.087840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:25.087840 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.087840 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:25.087840 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.087840 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.087840 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:25.087840 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.087840 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:25.088954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:25.089428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:25.089428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.089428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:25.089428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.089428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.089428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:25.089428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.089428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:25.090058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.090761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:25.091407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.091407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:25.167783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:25.168195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:25.169429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:25.170555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:25.171465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.172335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:25.172335 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.172335 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:25.172335 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.172335 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.172335 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:25.172335 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.172335 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.173495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:25.174533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:29.903596 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:29.904760 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:29.941958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:29.942208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:29.942901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:29.942901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:29.942901 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:29.942901 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:29.942901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:29.942901 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:29.942901 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:29.942901 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:29.944142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:29.944854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:30.074771 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:30.075749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:30.076004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:30.076189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.076884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:30.076884 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.076884 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:30.076884 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.076884 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.076884 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:30.076884 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.076884 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.078149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.078920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:30.082382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:30.082730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:30.083178 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:30.083379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.083850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:30.083850 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.083850 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:30.083850 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.083850 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.083850 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:30.083850 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.083850 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:30.084796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:30.085641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:30.085641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.085641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:30.085641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.085641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.085641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:30.085641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.085641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:30.086054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.087012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.087900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:30.087900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.087900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:30.087900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:30.087900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:30.087900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:30.087900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.088403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:30.088403 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.088403 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:30.088403 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.088403 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.088403 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:30.088403 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.088403 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:30.089264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:30.090135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:30.090135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.090135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:30.090135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.090135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.090135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:30.090135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.090135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:30.090506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.090931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:30.091414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.091414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:30.168320 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:30.168506 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:30.169287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:30.169806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:30.170077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.170862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:30.170862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.170862 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:30.170862 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.170862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.170862 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:30.170862 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.170862 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.172018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:30.172822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:34.904406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:34.905614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:34.940274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:34.940669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:34.941647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:34.941647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:34.941647 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:34.941647 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:34.941647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:34.941647 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:34.941647 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:34.941647 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:34.942998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:34.943833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:35.075351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:35.076421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:35.076696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:35.077031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.077814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:35.077814 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.077814 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:35.077814 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.077814 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.077814 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:35.077814 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.077814 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.079003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.079471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:35.082867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:35.083439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:35.084123 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:35.084419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.085140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:35.085140 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.085140 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:35.085140 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.085140 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.085140 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:35.085140 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.085140 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:35.086522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:35.087051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:35.087051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.087051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:35.087051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.087051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.087051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:35.087051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.087051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:35.088460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.088889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.089381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:35.089381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.089381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:35.089381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:35.089381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:35.089381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:35.089572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.090298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:35.090298 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.090298 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:35.090298 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.090298 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.090298 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:35.090298 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.090298 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:35.091708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:35.092167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:35.092167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.092167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:35.092167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.092167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.092167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:35.092167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.092167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:35.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.094687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:35.095371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.095371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:35.168833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:35.169283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:35.169983 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:35.170474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:35.170814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.172192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:35.172192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.172192 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:35.172192 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.172192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.172192 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:35.172192 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.172192 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.173798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:35.174284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:39.905266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:39.907021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:39.944718 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:39.944718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:39.945648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:39.945648 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:39.945648 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:39.945648 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:39.945648 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:39.945648 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:39.945648 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:39.945648 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:39.946708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:39.947708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:40.076023 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:40.077361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:40.077707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:40.078002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.078942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:40.078942 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.078942 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:40.078942 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.078942 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.078942 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:40.078942 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.078942 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.080656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.081493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:40.083393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:40.083730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:40.084192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:40.084487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.085182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:40.085182 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.085182 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:40.085182 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.085182 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.085182 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:40.085182 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.085182 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:40.086417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:40.087310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:40.087310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.087310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:40.087310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.087310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.087310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:40.087310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.087310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:40.088037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.089246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.090025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:40.090025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.090025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:40.090025 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:40.090025 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:40.090306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:40.090557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.090996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:40.090996 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.090996 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:40.090996 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.090996 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.090996 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:40.090996 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.090996 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:40.091887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:40.092433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:40.092433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.092433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:40.092433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.092433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.092433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:40.092433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.092433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:40.092896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.093336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:40.093786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.093786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:40.169495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:40.170096 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:40.170601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:40.170942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:40.171200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.171893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:40.171893 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.171893 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:40.171893 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.171893 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.171893 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:40.171893 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.171893 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.173693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:40.174606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:44.905869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:44.907508 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:44.942076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:44.942336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:44.943335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:44.943335 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:44.943335 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:44.943335 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:44.943335 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:44.943335 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:44.943335 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:44.943335 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:44.944634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:44.946373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:45.076661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:45.078462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:45.078796 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:45.079076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.081683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:45.082632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.083833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.083833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:45.083833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.083833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:45.084094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:45.084837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:45.084963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:45.085334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.085972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:45.085972 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.085972 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:45.085972 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.085972 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.085972 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:45.085972 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.085972 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:45.087251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.087928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:45.088780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.089366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:45.089949 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:45.090101 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:45.090797 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:45.091169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.092025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:45.092025 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.092025 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:45.092025 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.092025 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.092025 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:45.092025 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.092025 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:45.093363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:45.093733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:45.093733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.093733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:45.093733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.093733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.093733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:45.093733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.093733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:45.095044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.095391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:45.095728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.095728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:45.170066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:45.170639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:45.171233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:45.171500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:45.171653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.172431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:45.172431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.172431 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:45.172431 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.172431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.172431 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:45.172431 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.172431 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.174377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:45.175588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:49.906622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:49.908136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:49.943775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:49.944219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:49.945209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:49.945209 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:49.945209 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:49.945209 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:49.945209 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:49.945209 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:49.945209 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:49.945209 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:49.946641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:49.947238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:50.077626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:50.078537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:50.078718 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:50.078847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.079382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:50.079382 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.079382 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:50.079382 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.079382 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.079382 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:50.079382 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.079382 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.080423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.080770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:50.084381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:50.084900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:50.085303 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:50.085472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.086087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:50.086087 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.086087 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:50.086087 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.086087 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.086087 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:50.086087 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.086087 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:50.087295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:50.087669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:50.087669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.087669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:50.087669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.087669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.087669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:50.087669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.087669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:50.088752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.089107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.089446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:50.089446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.089446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:50.090472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:50.090869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:50.091224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:50.091442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.091878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:50.091878 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.091878 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:50.091878 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.091878 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.091878 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:50.091878 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.091878 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:50.092732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:50.093037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:50.093037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.093037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:50.093037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.093037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.093037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:50.093037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.093037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:50.093740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.094069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:50.094439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.094439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:50.170722 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:50.171476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:50.172057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:50.172522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:50.172748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.173659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:50.173659 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.173659 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:50.173659 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.173659 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.173659 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:50.173659 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.173659 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.176365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:50.176695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:54.907285 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:54.908325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:54.938204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:54.938363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:54.939192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:54.939192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:54.939192 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:54.939192 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:54.939192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:54.939192 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:54.939192 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:54.939192 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:54.939644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:54.940536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:55.078491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:55.079476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:55.079726 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:55.079939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.080796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:55.080796 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.080796 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:55.080796 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.080796 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.080796 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:55.080796 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.080796 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.082296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.082823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:55.084727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:55.085204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:07:55.085535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:55.085713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.086192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:55.086192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.086192 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:55.086192 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.086192 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.086192 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:55.086192 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.086192 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:55.087043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:55.087537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:55.087537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.087537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:55.087537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.087537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.087537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:55.087537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.087537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:07:55.087955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.088509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.089038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:55.089038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.089038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:55.090779 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:55.090998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:55.091261 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:55.091533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.091932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:07:55.091932 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.091932 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:55.091932 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.091932 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.091932 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:55.091932 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.091932 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:07:55.092710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:55.093507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:07:55.093507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.093507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:07:55.093507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.093507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.093507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:07:55.093507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.093507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:07:55.093873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.094686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:55.095499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.095499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:55.171251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:55.172097 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:55.172806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:55.172917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:55.173195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.174484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:07:55.174484 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.174484 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:55.174484 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.174484 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.174484 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:55.174484 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.174484 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.177254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:55.178109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:07:59.908120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:07:59.924638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:07:59.949195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:07:59.949564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:59.950456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:07:59.950456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:59.950456 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:07:59.950456 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:07:59.950456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:59.950456 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:07:59.950456 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:59.950456 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:59.951865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:07:59.952290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:00.079186 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:00.080154 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:00.080562 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:00.080883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.081724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:00.081724 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.081724 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:00.081724 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.081724 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.081724 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:00.081724 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.081724 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.083774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.084164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:00.085069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:00.085265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:00.085514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:00.085691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.086103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:00.086103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.086103 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:00.086103 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.086103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.086103 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:00.086103 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.086103 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:00.086787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:00.087156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:00.087156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.087156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:00.087156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.087156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.087156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:00.087156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.087156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:00.087756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.088089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.088427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:00.088427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.088427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:00.091116 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:00.091318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:00.091551 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:00.091768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.092175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:00.092175 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.092175 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:00.092175 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.092175 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.092175 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:00.092175 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.092175 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:00.092928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:00.093282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:00.093282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.093282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:00.093282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.093282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.093282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:00.093282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.093282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:00.094030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.094341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:00.094693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.094693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:00.171962 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:00.172541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:00.173000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:00.173263 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:00.173420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.174115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:08:00.174115 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.174115 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:00.174115 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.174115 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.174115 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:00.174115 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.174115 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.175829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:00.176636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:04.909058 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:04.910333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:04.949652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:04.949817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:04.950273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:04.950273 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:04.950273 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:04.950273 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:04.950273 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:04.950273 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:04.950273 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:04.950273 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:04.951049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:04.951641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:05.080229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:05.081541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:05.081703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:05.081791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.082242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:05.082242 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.082242 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:05.082242 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.082242 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.082242 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:05.082242 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.082242 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.083003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.083924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:05.085520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:05.085836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:05.086338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:05.086492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.086893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:05.086893 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.086893 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:05.086893 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.086893 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.086893 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:05.086893 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.086893 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:05.088199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:05.089089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:05.089089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.089089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:05.089089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.089089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.089089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:05.089089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.089089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:05.089799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.090393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.090903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:05.090903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.090903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:05.091461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:05.092032 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:05.092299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:05.092535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:05.094970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.095589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:05.095985 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.096510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:05.172816 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:05.173574 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:05.174168 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:05.174970 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:05.174970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.175636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:08:05.175636 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.175636 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:05.175636 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.175636 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.175636 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:05.175636 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.175636 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.178623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:05.179433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:09.909887 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:09.911104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:09.942222 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:09.942695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:09.943597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:09.943597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:09.943597 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:09.943597 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:09.943597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:09.943597 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:09.943597 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:09.943597 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:09.944525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:09.945319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:10.085382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:10.086589 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:10.087238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:10.087828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:10.088252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:10.088519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.089468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:10.089468 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.089468 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:10.089468 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.089468 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.089468 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:10.089468 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.089468 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.090992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.091632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.092181 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:10.092571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:10.093731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:10.094282 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:10.095914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.096920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.097890 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:10.098844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.100870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:10.101645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.102530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:10.173490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:10.173956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:10.174688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:10.175160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:10.175801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.176461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:08:10.176461 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.176461 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:10.176461 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.176461 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.176461 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:10.176461 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.176461 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.178296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:10.179089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:14.910471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:14.913448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:14.949063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:14.950766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:14.952776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:15.086098 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:15.087324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:15.087942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:15.088746 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:15.089022 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:15.089181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.089860 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:15.090066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.090653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:15.090653 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.090653 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:15.090653 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.090653 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.090653 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:15.090653 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.090653 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:15.091211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:15.091211 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.091211 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:15.091211 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.091211 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.091211 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:15.091211 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.091211 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.092871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.094053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:15.095225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:15.096341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.097390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:15.098103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.098652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.099257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.099830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:15.100468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.101213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.101213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:15.101213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.101213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:15.174077 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:15.174685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:15.175238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:15.175644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:15.175913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.176796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:08:15.176796 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.176796 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:15.176796 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.176796 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.176796 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:15.176796 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.176796 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.178404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:15.178864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:19.911088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:19.912664 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:19.950469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:19.950653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:19.951452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:19.951452 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:19.951452 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:19.951452 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:19.951452 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:19.951452 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:19.951452 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:19.951452 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:19.952660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:19.953329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:20.087039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:20.088287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:20.095063 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:20.095575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:20.098925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:20.099211 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:20.099354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.100117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:20.100117 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.100117 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:20.100117 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.100117 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.100117 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:20.100117 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.100117 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.101185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.102047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:20.104630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:20.105101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.105556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.105556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:20.105556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.105556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:20.106607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.107110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.107579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:20.107579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.107579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:20.107579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:20.111367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.112013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:20.112013 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.112013 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:20.112013 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.112013 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.112013 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:20.112013 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.112013 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:20.112543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:20.112949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:20.112949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.112949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:20.112949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.112949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.112949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:20.112949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.112949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:20.113251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.113727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:20.114376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.114376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:20.174743 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:20.175295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:20.175739 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:20.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:20.176408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.177155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:08:20.177155 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.177155 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:20.177155 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.177155 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.177155 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:20.177155 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.177155 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.178851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:20.179439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:24.804197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:24.808433 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=260 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:08:24.829128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 26 01:08:24.829559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=2, routers=1, router_ports=1 {{(pid=105681) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}}
Mar 26 01:08:24.829559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:08:24.859092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, resource={'id': 'f7e819e4-7101-412b-b047-baa5e16b2a52', 'name': 'admin_net', 'tenant_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'project_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f2e900ee-4d88-4151-8e50-661745950632'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:14Z', 'updated_at': '2026-03-26T01:02:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4803}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.859523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.860080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d43d91f1-f24a-411f-bae7-af85048f95b9, external_ids_update=None, columns={'tag': 4803}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.860256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.861188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:08:24.861188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:24.861188 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:24.861188 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:24.861188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:24.861188 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:24.861188 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.861188 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:24.862174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:08:24.863359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:24.863903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:08:24.863903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.863903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:08:24.863903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:08:24.891561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-db438906-c74f-4dc8-883c-6e56962e551b, resource={'id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'name': 'public', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['1c3e4583-6e63-4953-8050-3341662a5ec6', '23682989-62bf-4e07-a74f-0cb01077d5ac'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-26T00:58:16Z', 'updated_at': '2026-03-26T00:58:25Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.891919 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-db438906-c74f-4dc8-883c-6e56962e551b, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.892515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-200618ec-0540-41b0-9429-dc7dfc74abb0, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.892938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.893809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:08:24.893809 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:24.893809 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:24.893809 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:24.893809 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:24.893809 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:24.893809 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.893809 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:24.895104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:08:24.896002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:24.896896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:08:24.896896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.896896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:08:24.896896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:08:24.912249 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:24.915999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:24.937888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, resource={'id': 'f481c2f9-5052-46d5-821a-6c8712ec70ec', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['329ea1d3-f656-45b1-bf23-1d927d4578ea'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:50Z', 'updated_at': '2026-03-26T01:02:51Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19903}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.938409 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.939084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3aa5be7a-5c67-4f1a-bdf2-7bf6912e871d, external_ids_update=None, columns={'tag': 19903}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.939429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.945198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:08:24.945198 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:24.945198 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:24.945198 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:24.945198 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:24.945198 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:24.945198 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.945198 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:24.947404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:08:24.947757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:24.948125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:08:24.948125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.948125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:08:24.948125 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:08:24.984729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:24.985928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.988002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:24.988002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:24.988002 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:24.988002 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:24.988002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:24.988002 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:24.988002 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.988002 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:24.990323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:24.991147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:25.001366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, resource={'id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'name': 'private', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['02d96a4b-4c93-4b95-ad0f-f11d085f0699', '0503b55a-1b69-49b5-b418-16ba37be90e9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T00:58:10Z', 'updated_at': '2026-03-26T00:58:13Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4449}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.001688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.002093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6fe31530-ff9f-476b-aea1-4804059b76cd, external_ids_update=None, columns={'tag': 4449}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.002244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.002854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:08:25.002854 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.002854 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:25.002854 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.002854 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.002854 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:25.002854 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.002854 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:25.003786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.004113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.004501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:08:25.004501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.004501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:08:25.004501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource bdc08697-d233-4e65-b7e1-60c606976a9e (type: security_groups) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:08:25.087601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:25.088692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.089143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.089378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.090236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:25.090236 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.090236 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:25.090236 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.090236 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.090236 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:25.090236 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.090236 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.091147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.091876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:25.095573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:25.096016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:25.096579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.096959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.097779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:25.097779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.097779 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:25.097779 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.097779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.097779 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:25.097779 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.097779 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:25.099130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.099779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:25.099779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.099779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:25.099779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.099779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.099779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:25.099779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.099779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:25.100795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.122246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.122893 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:25.123326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:25.123900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.124312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.124935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:25.173146 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.db.ovn_revision_numbers_db [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Successfully bumped revision number for resource bdc08697-d233-4e65-b7e1-60c606976a9e (type: security_groups) to 15
Mar 26 01:08:25.173523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource c4541653-f9a3-4566-9b4f-26d882905d0f (type: security_groups) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:08:25.175920 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:25.176674 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:25.177382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.177653 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.177807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.181562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:08:25.181562 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.181562 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:25.181562 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.181562 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.181562 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:25.181562 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.181562 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.184392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.185053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:25.236325 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.db.ovn_revision_numbers_db [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Successfully bumped revision number for resource c4541653-f9a3-4566-9b4f-26d882905d0f (type: security_groups) to 10
Mar 26 01:08:25.236518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:08:25.351250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, resource={'id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'enable_snat': True}, 'gw_port_id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-ryt6n', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'ip_address': '172.24.5.142', 'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac'}, {'ip_address': '2001:db8::16a', 'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 3, 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8'}, resource_type=routers, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.351457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.351542 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.351643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.351730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Unable to update router e4efd798-3c25-4ea9-bd45-fa7a05aeaef9. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:08:25.355003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.355406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:08:25.355799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.356227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.356227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:08:25.356227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.356227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:08:25.356227 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Fixing resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:08:25.508485 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, resource={'id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'name': '', 'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'project_id': '', 'mac_address': 'fa:16:3e:4e:d4:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160849', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.508819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::16a/64', '172.24.5.142/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.509236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=14a0aa42-40aa-4508-9af3-cffdffef9818, lrouter_port=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:25.509616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Traceback (most recent call last):
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Failed to fix resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:08:25.511595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:08:25.512024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:08:25.512368 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-454c2005-1f4a-4cc0-a9fb-9e5f2eb07aed None None] Maintenance task: Synchronization completed (took 0.68 seconds)
Mar 26 01:08:29.913010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:29.914580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:29.944610 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:29.945049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:29.945883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:29.945883 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:29.945883 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:29.945883 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:29.945883 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:29.945883 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:29.945883 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:29.945883 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:29.947031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:29.948242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:30.088259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:30.089328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:30.089509 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:30.089629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.090302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:30.090302 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.090302 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:30.090302 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.090302 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.090302 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:30.090302 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.090302 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.091991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.092499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:30.096115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:30.096447 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:30.096710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:30.096858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.097218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:30.097218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.097218 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:30.097218 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.097218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.097218 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:30.097218 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.097218 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:30.097958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:30.098402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:30.098402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.098402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:30.098402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.098402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.098402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:30.098402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.098402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:30.098969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.099516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.100028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:30.100028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.100028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:30.101895 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:30.102219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:30.102498 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:30.102652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.103004 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:30.103651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:30.103965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:30.103965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.103965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:30.103965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.103965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.103965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:30.103965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.103965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:30.104385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.104710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:30.105374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.105374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:30.176805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:30.177420 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:30.177948 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:30.178363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:30.178595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.179449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:30.179449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.179449 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:30.179449 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.179449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.179449 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:30.179449 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.179449 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.180679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:30.181422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:34.913898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:34.914868 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:34.953368 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:34.953701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:34.954416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:34.954416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:34.954416 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:34.954416 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:34.954416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:34.954416 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:34.954416 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:34.954416 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:34.955181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:34.955904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:35.089197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:35.090305 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:35.090766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:35.091097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.092049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:35.092049 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.092049 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:35.092049 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.092049 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.092049 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:35.092049 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.092049 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.093847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.094460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:35.096458 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:35.097050 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:35.097471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:35.097839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.098604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:35.098604 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.098604 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:35.098604 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.098604 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.098604 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:35.098604 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.098604 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:35.099542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:35.100222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:35.100222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.100222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:35.100222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.100222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.100222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:35.100222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.100222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:35.100554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.100965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.101461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:35.101461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.101461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:35.102173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:35.102486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:35.102926 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:35.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.104246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:35.104246 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.104246 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:35.104246 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.104246 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.104246 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:35.104246 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.104246 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:35.106043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:35.106912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:35.106912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.106912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:35.106912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.106912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.106912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:35.106912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.106912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:35.107998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.108430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:35.108890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.108890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:35.177594 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:35.178160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:35.178934 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:35.179829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:35.179829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.180024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:35.180024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.180024 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:35.180024 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.180024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.180024 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:35.180024 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.180024 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.181297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:35.181826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:39.914643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:39.916676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:39.956380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:39.956844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:39.957888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:39.957888 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:39.957888 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:39.957888 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:39.957888 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:39.957888 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:39.957888 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:39.957888 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:39.959067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:39.959605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:40.093020 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:40.093701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:40.094036 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:40.094268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.095154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:40.095154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.095154 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:40.095154 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.095154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.095154 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:40.095154 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.095154 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.097196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.097576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:40.098572 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:40.099275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:40.100127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:40.100431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.101229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:40.101229 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.101229 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:40.101229 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.101229 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.101229 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:40.101229 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.101229 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:40.102051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:40.102401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:40.102401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.102401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:40.102401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.102401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.102401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:40.102401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.102401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:40.102893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:40.104252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.104612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.104943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:40.104943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.104943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:40.104943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:40.105051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:40.105357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.106067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:40.106067 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.106067 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:40.106067 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.106067 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.106067 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:40.106067 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.106067 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:40.107439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:40.107723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:40.107723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.107723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:40.107723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.107723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.107723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:40.107723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.107723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:40.109316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.109642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:40.109954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.109954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:40.178362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:40.179136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:40.179660 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:40.180093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:40.180347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.181235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:40.181235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.181235 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:40.181235 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.181235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.181235 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:40.181235 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.181235 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.182139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:40.182579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:44.916778 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:44.918472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:44.947283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:44.947548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:44.948347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:44.948347 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:44.948347 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:44.948347 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:44.948347 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:44.948347 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:44.948347 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:44.948347 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:44.949881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:44.950417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:45.093074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:45.093951 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:45.094150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:45.094289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.095265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:45.095265 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.095265 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:45.095265 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.095265 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.095265 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:45.095265 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.095265 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.096276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.096753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:45.099227 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:45.099748 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:45.099832 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:45.100055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.100478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:45.100478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.100478 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:45.100478 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.100478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.100478 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:45.100478 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.100478 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:45.103404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.103959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:45.104508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:45.105060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:45.105896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:45.106366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:45.106603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:45.110438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.110885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:45.111600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.111944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:45.112639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.112639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:45.179001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:45.179647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:45.180055 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:45.180321 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:45.182320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.183174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:45.183174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.183174 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:45.183174 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.183174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.183174 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:45.183174 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.183174 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.183962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:45.184626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:49.917518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:49.918837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:49.950222 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:49.950649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:49.951482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:49.951482 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:49.951482 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:49.951482 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:49.951482 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:49.951482 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:49.951482 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:49.951482 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:49.952374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:49.953258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:50.093520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:50.094647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:50.094855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:50.095210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.096856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:50.096856 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.096856 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:50.096856 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.096856 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.096856 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:50.096856 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.096856 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.098561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.098896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:50.099571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:50.099820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:50.100291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:50.100808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.101354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:50.101354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.101354 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:50.101354 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.101354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.101354 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:50.101354 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.101354 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:50.102232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:50.102697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:50.102697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.102697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:50.102697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.102697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.102697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:50.102697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.102697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:50.103250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.103623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.103970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:50.103970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.103970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:50.105712 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:50.106217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:50.106578 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:50.106767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.107185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:50.107185 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.107185 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:50.107185 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.107185 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.107185 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:50.107185 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.107185 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:50.108415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:50.108758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:50.108758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.108758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:50.108758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.108758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.108758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:50.108758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.108758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:50.109751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.110141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:50.110520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.110520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:50.179548 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:50.179987 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:50.180490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:50.181216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:50.181828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.182618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:50.182618 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.182618 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:50.182618 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.182618 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.182618 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:50.182618 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.182618 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.183701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:50.184097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:54.918441 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:54.919388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:54.961326 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:54.961526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:54.962196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:54.962196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:54.962196 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:54.962196 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:54.962196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:54.962196 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:54.962196 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:54.962196 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:54.963496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:54.964485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:55.093994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:55.107684 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:55.108257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:55.109443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:08:55.111736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.125462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.126255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:55.127269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:55.128033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.128916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:55.129785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:55.130465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.132134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.132828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.133451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:55.133850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:55.134333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.134333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:55.134333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.134333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.134333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:55.134333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.134333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:55.180340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:55.180801 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:55.181368 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:55.181737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:55.182007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:08:55.182803 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.182803 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:55.182803 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.182803 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.182803 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:55.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.184857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:55.185606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:08:59.918950 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:08:59.920603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:08:59.956892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:08:59.957062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:59.957489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:08:59.957489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:59.957489 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:08:59.957489 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:08:59.957489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:59.957489 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:08:59.957489 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:59.957489 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:59.958332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:08:59.959084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:00.094574 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:00.095762 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:00.096061 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:00.096260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.097050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:00.097050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.097050 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:00.097050 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.097050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.097050 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:00.097050 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.097050 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.098318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.099010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:00.108878 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:00.109558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:00.110312 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:00.110411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.110918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:00.110918 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.110918 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:00.110918 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.110918 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.110918 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:00.110918 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.110918 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:00.111609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:00.111943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:00.111943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.111943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:00.111943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.111943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.111943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:00.111943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.111943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:00.112410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.112733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.113078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:00.113078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.113078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:00.113078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:00.113078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:00.113421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:00.113859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.114409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:00.114409 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.114409 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:00.114409 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.114409 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.114409 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:00.114409 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.114409 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:00.114900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:00.115203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:00.115203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.115203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:00.115203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.115203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.115203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:00.115203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.115203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:00.115591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.115881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:00.116230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.116230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:00.181032 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:00.181448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:00.182012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:00.182298 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:00.182523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.183058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:00.183058 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.183058 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:00.183058 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.183058 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.183058 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:00.183058 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.183058 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.185258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:00.185776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:04.984754 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:04.984754 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:05.008745 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:05.009034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.010023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:09:05.010023 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.010023 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:05.010023 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.010023 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.010023 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:05.010023 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.010023 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.011100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.011657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:05.095001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:05.096185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:05.096458 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:05.096663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.097386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:05.097386 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.097386 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:05.097386 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.097386 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.097386 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:05.097386 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.097386 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.099099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.099665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:05.109383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:05.109599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:05.109966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:05.110187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.110669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:05.110669 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.110669 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:05.110669 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.110669 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.110669 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:05.110669 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.110669 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:05.111468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:05.112010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:05.112010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.112010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:05.112010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.112010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.112010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:05.112010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.112010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:05.113349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.113849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.114343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:05.114343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.114343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:05.114343 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:05.114343 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:05.114520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:05.115048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.115622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:05.115622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.115622 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:05.115622 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.115622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.115622 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:05.115622 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.115622 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:05.116153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:05.116565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:05.116565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.116565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:05.116565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.116565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.116565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:05.116565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.116565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:05.116819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.117271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:05.117817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.117817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:05.181364 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:05.181827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:05.182661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:05.182661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:05.182661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.183165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:05.183165 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.183165 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:05.183165 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.183165 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.183165 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:05.183165 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.183165 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.184465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:05.184944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:09.972390 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:09.973368 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:10.007622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:10.007897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.008633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:09:10.008633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.008633 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:10.008633 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.008633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.008633 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:10.008633 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.008633 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.009774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.010532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:10.095705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:10.097229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:10.097416 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:10.097622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.098952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:10.098952 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.098952 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:10.098952 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.098952 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.098952 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:10.098952 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.098952 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.100130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.100631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:10.109766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:10.110054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:10.110365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:10.110563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.110988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:10.110988 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.110988 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:10.110988 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.110988 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.110988 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:10.110988 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.110988 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:10.113868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.114521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:10.114975 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:10.115615 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.116054 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:10.117067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:10.117560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:10.117560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.117560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:10.117560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.117560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.117560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:10.117560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.117560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:10.118665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.119164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:10.119670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.119670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:10.182061 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:10.182457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:10.182948 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:10.183236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:10.183355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.184070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:10.184070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.184070 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:10.184070 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.184070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.184070 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:10.184070 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.184070 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.184742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:10.185562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:14.972894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:14.974442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:15.015558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:15.015864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.016837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:09:15.016837 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.016837 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:15.016837 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.016837 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.016837 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:15.016837 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.016837 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.018690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:15.096916 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:15.098427 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:15.098734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:15.099137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.100111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:15.100111 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.100111 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:15.100111 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.100111 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.100111 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:15.100111 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.100111 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.101458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.102236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:15.110260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:15.110894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:15.111346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:15.111591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.112401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:15.112401 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.112401 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:15.112401 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.112401 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.112401 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:15.112401 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.112401 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:15.113653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:15.114183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:15.114183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.114183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:15.114183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.114183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.114183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:15.114183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.114183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:15.115709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.116155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.116532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:15.116532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.116532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:15.116532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:15.116532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:15.116990 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:15.117394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.118164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:15.118164 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.118164 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:15.118164 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.118164 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.118164 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:15.118164 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.118164 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:15.119591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:15.120005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:15.120005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.120005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:15.120005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.120005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.120005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:15.120005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.120005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:15.121137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.121632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:15.122180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.122180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:15.182517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.204149 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.205277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:15.206235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:19.973576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:19.974602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:20.008860 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:20.009301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.010123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:09:20.010123 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.010123 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:20.010123 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.010123 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.010123 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:20.010123 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.010123 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.011027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.011538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:20.116069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:20.116240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:20.116311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:20.117241 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:20.117823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:20.118269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:20.118456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:20.118570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.119060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:20.119516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:20.119516 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.119516 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:20.119516 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.119516 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.119516 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:20.119516 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.119516 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:20.119779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.120999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.121805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:20.122309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:20.122916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.123480 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:20.124205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.124672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.125220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:20.125220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.125220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:20.140672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.141754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:20.142585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.143705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:20.183038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:20.183338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:20.183933 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:20.183933 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:20.184117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.184586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:20.184586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.184586 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:20.184586 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.184586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.184586 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:20.184586 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.184586 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.185552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:20.186017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:24.974103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:24.975538 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:24.977320 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=471,neutron:UPDATE=2 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:09:25.028088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:25.028596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.029640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:09:25.029640 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.029640 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:25.029640 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.029640 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.029640 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:25.029640 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.029640 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.031005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.032170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:25.116872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:25.117113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:25.117996 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:25.118269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:25.118716 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:25.119352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:25.119643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:25.119869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.121065 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:25.121351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.121993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:25.121993 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.121993 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:25.121993 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.121993 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.121993 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:25.121993 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.121993 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:25.122832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:25.123178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:25.123178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.123178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:25.123178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.123178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.123178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:25.123178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.123178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:25.124086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.124434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:25.124801 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.126413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.126755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:25.127138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:25.129940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.130317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:25.130934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.131236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:25.131550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.131550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:25.183534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:25.183674 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:25.183908 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:25.184127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:25.184240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.184691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:25.184691 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.184691 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:25.184691 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.184691 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.184691 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:25.184691 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.184691 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.185480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:25.186267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:29.974799 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:29.976811 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:30.007891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:30.008255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.008923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:09:30.008923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.008923 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:30.008923 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.008923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.008923 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:30.008923 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.008923 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.009763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.010559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:30.117733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:30.119127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:30.119371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:30.119622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.120300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:30.120300 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.120300 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:30.120300 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.120300 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.120300 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:30.120300 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.120300 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.122378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.123214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:30.124705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:30.124705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:30.124705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:30.124705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:30.124705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:30.124705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.125229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:30.125441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.125878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:30.125878 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.125878 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:30.125878 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.125878 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.125878 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:30.125878 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.125878 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:30.126555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.127229 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:30.127901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.128639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:30.129473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:30.130687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.131081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.131464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:30.183994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:30.184805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:30.185654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:30.186120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:30.186601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.187745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:30.187745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.187745 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:30.187745 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.187745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.187745 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:30.187745 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.187745 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.189915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:30.190724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:34.975498 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:34.976258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:35.010005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:35.010465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.011325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:09:35.011325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.011325 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:35.011325 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.011325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.011325 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:35.011325 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.011325 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.012182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.012780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:35.118337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:35.119820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:35.120274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:35.120600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.121715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:35.121715 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.121715 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:35.121715 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.121715 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.121715 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:35.121715 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.121715 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.122962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.123528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:35.124008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:35.124008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:35.124377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:35.124590 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:35.125192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:35.125441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.126048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:35.126048 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.126048 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:35.126048 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.126048 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.126048 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:35.126048 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.126048 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:35.127323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:35.127991 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:35.128799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.129552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.130084 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:35.130426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:35.130872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:35.130872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.130872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:35.130872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.130872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.130872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:35.130872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.130872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:35.131360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.131793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:35.132375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.132375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:35.184693 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:35.185317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:35.185771 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:35.186194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:35.186429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.187296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:35.187296 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.187296 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:35.187296 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.187296 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.187296 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:35.187296 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.187296 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.188958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:35.192302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:39.976029 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:39.977890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:40.011949 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:40.012180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.012833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:09:40.012833 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.012833 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:40.012833 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.012833 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.012833 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:40.012833 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.012833 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.014558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.015941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:40.119084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:40.120384 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:40.120524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:40.120831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.121517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:40.121517 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.121517 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:40.121517 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.121517 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.121517 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:40.121517 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.121517 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.122793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.123352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:40.124502 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:40.124602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:40.124814 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:40.125207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:40.125423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.125760 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:40.126288 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:40.126566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.127050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:40.127050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.127050 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:40.127050 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.127050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.127050 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:40.127050 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.127050 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:40.127940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.128661 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:40.129157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.129730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:40.130458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.130939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:40.131273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.131725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.132369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:40.132369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.132369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:40.185402 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:40.185945 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:40.186499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:40.186920 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:40.186920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.187564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:40.187564 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.187564 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:40.187564 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.187564 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.187564 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:40.187564 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.187564 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.189285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:40.190293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:44.976637 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:44.977813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:45.016094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:45.016489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.017321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:09:45.017321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.017321 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:45.017321 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.017321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.017321 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:45.017321 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.017321 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.018535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.019298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:45.119713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:45.121633 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:45.121998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:45.122362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.123271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:45.123271 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.123271 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:45.123271 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.123271 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.123271 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:45.123271 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.123271 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.124427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.125276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:45.126240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:45.126240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:45.126240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:45.129478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:45.129717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.130359 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:45.132117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:45.132117 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.132117 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:45.132117 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.132117 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.132117 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:45.132117 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.132117 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:45.132517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:45.133467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:45.133926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.134471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:45.138406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:45.138899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:45.138899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.138899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:45.138899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.138899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.138899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:45.138899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.138899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:45.139380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.139878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:45.140426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.140426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:45.185986 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:45.186472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:45.186969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:45.187333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:45.187543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.188852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:45.188852 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.188852 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:45.188852 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.188852 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.188852 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:45.188852 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.188852 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.189941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:45.190497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:49.977181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:49.979429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:50.015338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:50.015527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.016258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:09:50.016258 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.016258 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:50.016258 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.016258 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.016258 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:50.016258 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.016258 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.017461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.017871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:50.120969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:50.122188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:50.122443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:50.122619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.123340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:50.123340 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.123340 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:50.123340 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.123340 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.123340 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:50.123340 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.123340 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.125065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.125911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:50.126931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:50.126931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:50.126931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:50.126931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.127200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:50.127200 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.127200 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:50.127200 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.127200 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.127200 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:50.127200 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.127200 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:50.127950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:50.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:50.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:50.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:50.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.128824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:50.129302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.130105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.130983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:50.130983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.130983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:50.130983 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:50.130983 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:50.132174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:50.132948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.134282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:50.134282 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.134282 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:50.134282 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.134282 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.134282 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:50.134282 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.134282 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:50.135612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:50.136052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:50.136052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.136052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:50.136052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.136052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.136052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:50.136052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.136052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:50.137553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.138041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:50.138545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.138545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:50.186579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:50.187219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:50.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:50.188036 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:50.188036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.188499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:50.188499 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.188499 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:50.188499 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.188499 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.188499 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:50.188499 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.188499 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.189839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:50.190484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:54.978819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:54.981072 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:55.018217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:55.018401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.019036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:09:55.019036 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.019036 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:55.019036 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.019036 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.019036 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:55.019036 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.019036 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.020300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.021030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:55.121658 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:55.123192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:55.123740 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:55.124056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.124804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:55.124804 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.124804 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:55.124804 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.124804 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.124804 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:55.124804 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.124804 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.127507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.130316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.131475 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:55.133609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.135190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:55.137252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:55.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.140517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.141083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.142602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:55.143269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.144129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.144129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:55.144129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.144129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:55.189376 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:55.189376 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:09:55.189920 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:55.190358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:09:55.190601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.191745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:09:55.191745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.191745 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:09:55.191745 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.191745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.191745 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:09:55.191745 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.191745 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.192489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:09:55.193426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:09:59.979429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:09:59.980630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:00.024534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:00.024841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.025558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:00.025558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.025558 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:00.025558 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.025558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.025558 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:00.025558 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.025558 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.026718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.027378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:00.122195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:00.123344 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:00.123448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:00.123848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.124341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:00.124341 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.124341 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:00.124341 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.124341 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.124341 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:00.124341 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.124341 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.126140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.126949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:00.127799 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:00.127799 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:00.127923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:00.128184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.128861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:00.128861 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.128861 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:00.128861 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.128861 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.128861 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:00.128861 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.128861 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:00.130533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:00.131325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:00.131325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.131325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:00.131325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.131325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.131325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:00.131325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.131325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:00.132893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.133621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.134372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:00.134372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.134372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:00.134372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:00.134372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:00.134372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:00.134668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.135553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:00.135553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.135553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:00.135553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.135553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.135553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:00.135553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.135553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:00.137551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:00.138236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:00.138236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.138236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:00.138236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.138236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.138236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:00.138236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.138236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:00.138723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.139616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:00.140362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.140362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:00.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:00.190252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:00.190793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:00.191196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:00.191403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.192206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:00.192206 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.192206 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:00.192206 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.192206 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.192206 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:00.192206 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.192206 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.193598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:00.194154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:04.979999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:04.981667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:05.012022 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:05.012219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.012939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:05.012939 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.012939 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:05.012939 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.012939 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.012939 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:05.012939 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.012939 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.014137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.014554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:05.123134 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:05.124089 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:05.124419 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:05.124768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.125860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:05.125860 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.125860 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:05.125860 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.125860 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.125860 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:05.125860 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.125860 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.127919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:05.129179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:05.129179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:05.129280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:05.129525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.130176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:05.130176 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.130176 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:05.130176 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.130176 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.130176 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:05.130176 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.130176 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:05.131179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:05.131727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:05.131727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.131727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:05.131727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.131727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.131727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:05.131727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.131727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:05.133317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.133963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.134515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:05.134515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.134515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:05.134515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:05.134659 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:05.135115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:05.135343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:05.137652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.138266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:05.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.139502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:05.140499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.140499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:05.190257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:05.190670 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:05.191064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:05.191283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:05.191398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.191832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:05.191832 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.191832 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:05.191832 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.191832 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.191832 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:05.191832 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.191832 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.192755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:05.193285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:09.981294 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:09.983621 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:10.017001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:10.017233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.017723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:10.017723 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.017723 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:10.017723 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.017723 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.017723 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:10.017723 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.017723 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.018615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.019475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:10.123459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:10.124932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:10.125188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:10.125474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.126456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:10.126456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.126456 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:10.126456 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.126456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.126456 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:10.126456 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.126456 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.128286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.129134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:10.129872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:10.129872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:10.129872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:10.130253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.131007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:10.131007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.131007 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:10.131007 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.131007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.131007 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:10.131007 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.131007 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:10.132342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:10.133165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:10.133165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.133165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:10.133165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.133165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.133165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:10.133165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.133165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:10.133559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.134553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.135415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:10.135415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.135415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:10.135415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:10.135415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:10.135415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:10.135415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.135843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:10.135843 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.135843 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:10.135843 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.135843 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.135843 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:10.135843 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.135843 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:10.136642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:10.137074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:10.137074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.137074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:10.137074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.137074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.137074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:10.137074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.137074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:10.137364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.137817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:10.138328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.138328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:10.190786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:10.191214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:10.191898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:10.192219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:10.192369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.192937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:10.192937 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.192937 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:10.192937 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.192937 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.192937 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:10.192937 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.192937 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.194832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:10.195403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:14.982229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:14.983306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:15.020233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:15.020450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.021024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:15.021024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.021024 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:15.021024 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.021024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.021024 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:15.021024 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.021024 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.022733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.023213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:15.124159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:15.125050 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:15.125423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:15.125423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.126035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:15.126035 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.126035 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:15.126035 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.126035 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.126035 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:15.126035 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.126035 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.126957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.127766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:15.129366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:15.131299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:15.131560 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:15.131703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.132127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:15.132127 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.132127 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:15.132127 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.132127 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.132127 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:15.132127 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.132127 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:15.132667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.133543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:15.134547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:15.135399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.138069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.138519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.138793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:15.139180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.139521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.139521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:15.139521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.139521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:15.191477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:15.192079 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:15.192371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:15.192585 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:15.192734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.193123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:15.193123 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.193123 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:15.193123 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.193123 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.193123 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:15.193123 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.193123 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.193831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:15.194300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:19.983499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:19.984643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:20.029393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:20.029806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.030718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:20.030718 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.030718 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:20.030718 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.030718 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.030718 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:20.030718 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.030718 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.031948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.034386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:20.124717 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:20.125824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:20.126440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:20.126686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.127653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:20.127653 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.127653 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:20.127653 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.127653 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.127653 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:20.127653 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.127653 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.128933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.130950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:20.132163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:20.132163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:20.132163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:20.132295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.132658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:20.132658 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.132658 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:20.132658 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.132658 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.132658 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:20.132658 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.132658 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:20.133508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:20.134138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:20.134138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.134138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:20.134138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.134138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.134138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:20.134138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.134138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:20.134610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.135250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.135873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:20.135873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.135873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:20.135873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:20.135873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:20.136285 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:20.136497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.136990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:20.136990 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.136990 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:20.136990 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.136990 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.136990 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:20.136990 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.136990 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:20.138155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:20.138725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:20.138725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.138725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:20.138725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.138725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.138725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:20.138725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.138725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:20.140495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.140899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:20.142745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.142745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:20.192187 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:20.192786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:20.193428 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:20.193785 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:20.194078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.194793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:20.194793 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.194793 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:20.194793 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.194793 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.194793 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:20.194793 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.194793 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.196554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:20.197483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:24.983588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:24.985727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:25.012905 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=256 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:10:25.021483 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:25.021763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.022431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:25.022431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.022431 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:25.022431 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.022431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.022431 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:25.022431 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.022431 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.023441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.024143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:25.125337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:25.126321 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:25.126569 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:25.126722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.127427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:25.127427 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.127427 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:25.127427 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.127427 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.127427 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:25.127427 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.127427 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.128338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.128819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:25.132131 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:25.132493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:25.132834 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:25.133039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.133466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:25.133466 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.133466 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:25.133466 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.133466 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.133466 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:25.133466 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.133466 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:25.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:25.134626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:25.134626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.134626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:25.134626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.134626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.134626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:25.134626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.134626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:25.135224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.136043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.136524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:25.136524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.136524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:25.136524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:25.136524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:25.136809 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:25.137103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.137865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:25.137865 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.137865 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:25.137865 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.137865 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.137865 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:25.137865 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.137865 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:25.139372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:25.139828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:25.139828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.139828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:25.139828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.139828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.139828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:25.139828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.139828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:25.141490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.142206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:25.142957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.142957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:25.192805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:25.193172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:25.193710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:25.193894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:25.194029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.194509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:25.194509 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.194509 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:25.194509 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.194509 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.194509 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:25.194509 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.194509 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.195633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:25.196187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:29.984665 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:29.986143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:30.022900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:30.023164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.024111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:10:30.024111 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.024111 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:30.024111 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.024111 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.024111 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:30.024111 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.024111 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.025222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.026270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:30.125866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:30.126967 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:30.127152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:30.127299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.127726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:30.127726 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.127726 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:30.127726 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.127726 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.127726 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:30.127726 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.127726 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.128473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.129108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:30.132446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:30.132610 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:30.132804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:30.132945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.133323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:30.133323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.133323 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:30.133323 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.133323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.133323 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:30.133323 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.133323 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:30.133791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:30.134314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:30.134314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.134314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:30.134314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.134314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.134314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:30.134314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.134314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:30.134564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.135028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.135578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:30.135578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.135578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:30.136278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:30.136708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:30.136987 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:30.137173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.137514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:30.137514 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.137514 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:30.137514 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.137514 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.137514 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:30.137514 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.137514 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:30.138378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:30.138845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:30.138845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.138845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:30.138845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.138845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.138845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:30.138845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.138845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:30.139472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.139915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:30.140440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.140440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:30.193233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:30.193650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:30.194194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:30.194476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:30.194624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.195446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:30.195446 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.195446 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:30.195446 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.195446 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.195446 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:30.195446 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.195446 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.196314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:30.196769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:34.985260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:34.987001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:35.032464 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:35.032667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.033393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:35.033393 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.033393 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:35.033393 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.033393 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.033393 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:35.033393 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.033393 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.034719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.035268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:35.126596 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:35.127885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:35.128472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:35.128472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.129277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:35.129277 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.129277 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:35.129277 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.129277 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.129277 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:35.129277 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.129277 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.130911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.131541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:35.132834 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:35.133338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:35.134149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:35.134595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.135394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:35.135394 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.135394 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:35.135394 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.135394 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.135394 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:35.135394 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.135394 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:35.136628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:35.137128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:35.137128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.137128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:35.137128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.137128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.137128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:35.137128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.137128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:35.138132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.138605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.139192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:35.139192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.139192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:35.139192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:35.139720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:35.139720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:35.140060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.140805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:35.140805 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.140805 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:35.140805 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.140805 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.140805 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:35.140805 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.140805 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:35.141255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:35.141819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:35.141819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.141819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:35.141819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.141819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.141819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:35.141819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.141819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:35.142089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.142697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:35.143315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.143315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:35.193807 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:35.194353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:35.195040 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:35.195381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:35.195572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.196275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:35.196275 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.196275 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:35.196275 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.196275 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.196275 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:35.196275 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.196275 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:35.198072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:39.985876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:39.987013 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:40.021700 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:40.021963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.022680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:40.022680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.022680 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:40.022680 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.022680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.022680 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:40.022680 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.022680 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.023535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.023999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:40.127299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:40.128484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:40.128718 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:40.128878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.129521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:40.129521 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.129521 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:40.129521 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.129521 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.129521 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:40.129521 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.129521 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.131227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.131636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:40.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:40.133525 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:40.133874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:40.134099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.134586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:40.134586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.134586 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:40.134586 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.134586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.134586 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:40.134586 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.134586 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:40.135797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:40.136181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:40.136181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.136181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:40.136181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.136181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.136181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:40.136181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.136181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:40.137565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.137890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.138300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:40.138300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.138300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:40.139397 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:40.139627 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:40.140167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:40.140386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.140844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:40.140844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.140844 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:40.140844 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.140844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.140844 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:40.140844 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.140844 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:40.141582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:40.141918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:40.141918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.141918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:40.141918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.141918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.141918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:40.141918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.141918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:40.142571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.142926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:40.143416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.143416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:40.194390 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:40.194773 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:40.195286 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:40.195618 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:40.195842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.196517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:40.196517 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.196517 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:40.196517 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.196517 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.196517 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:40.196517 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.196517 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.197410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:40.197992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:44.986358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:44.987238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:45.020049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:45.020400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.021128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:45.021128 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.021128 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:45.021128 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.021128 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.021128 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:45.021128 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.021128 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.022041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.022531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:45.128048 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:45.129178 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:45.129503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:45.129743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.130869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:45.130869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.130869 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:45.130869 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.130869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.130869 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:45.130869 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.130869 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.132855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.133245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:45.133837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:45.133837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:45.134557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:45.135085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.135948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:45.135948 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.135948 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:45.135948 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.135948 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.135948 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:45.135948 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.135948 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:45.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:45.138336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:45.138336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.138336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:45.138336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.138336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.138336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:45.138336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.138336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:45.138894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.139642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.140713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:45.140713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.140713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:45.140713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:45.141066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:45.141369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:45.141599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.142248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:45.142248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.142248 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:45.142248 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.142248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.142248 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:45.142248 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.142248 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:45.142919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:45.143386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:45.143386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.143386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:45.143386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.143386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.143386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:45.143386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.143386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:45.143660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.144262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:45.144784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.144784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:45.195004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:45.195615 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:45.196317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:45.196579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:45.196750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.197432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:45.197432 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.197432 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:45.197432 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.197432 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.197432 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:45.197432 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.197432 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.199239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:45.199607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:49.987185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:49.988800 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:50.022603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:50.023181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.023843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:50.023843 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.023843 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:50.023843 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.023843 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.023843 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:50.023843 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.023843 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.025123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.026509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:50.128676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:50.142232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:50.142389 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:50.144033 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:50.144033 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:50.144033 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:50.144250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:50.144349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.145212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:10:50.145212 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.145212 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:50.145212 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.145212 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.145212 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:50.145212 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.145212 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:50.145827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:50.147923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.148504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.149819 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:50.150963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:50.152287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:50.152734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:50.153303 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.153838 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:50.153838 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.153838 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.153838 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:50.153838 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.153838 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:50.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:50.154992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:50.154992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.154992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:50.154992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.154992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.154992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:50.154992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.154992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:50.156639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.157177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:50.157705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.157705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:50.195709 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:50.196327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:50.196859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:50.197070 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:50.197230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.197723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:50.197723 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.197723 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:50.197723 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.197723 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.197723 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:50.197723 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.197723 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.198997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:50.202185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:54.988454 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:54.989365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.029578 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.030268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.030909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:55.129323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:55.131600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:55.131822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:55.131944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.132607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:10:55.132607 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.132607 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:55.132607 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.132607 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.132607 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:55.132607 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.132607 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:55.133528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.134007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:55.142821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:55.187053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:55.187053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:10:55.187053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:55.187053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:55.187053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.187053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:55.187673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:55.189558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:55.190458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:55.191059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:55.191516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:55.192332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.193001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.193798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:10:55.196336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:10:55.196830 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:10:55.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:55.197892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:10:55.198208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.199059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:10:55.199059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.199059 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:10:55.199059 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.199059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.199059 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:10:55.199059 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.199059 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.199737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:10:55.200521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:00.008157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:00.008157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:00.022862 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:00.023314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.024238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:00.024238 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.024238 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:00.024238 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.024238 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.024238 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:00.024238 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.024238 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.025266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.025963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:00.131434 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:00.132858 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:00.133331 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:00.133580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.134423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:00.134423 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.134423 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:00.134423 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.134423 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.134423 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:00.134423 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.134423 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:00.136261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.137449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:00.143276 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:00.143441 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:00.143685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:00.144238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:00.144584 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:00.144800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.145431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:00.145677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.146010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:00.146010 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.146010 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:00.146010 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.146010 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.146010 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:00.146010 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.146010 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:00.146901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:00.147788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:00.147788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.147788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:00.147788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.147788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.147788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:00.147788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.147788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:00.148325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.149108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:00.149959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.150820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:00.151560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.152478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:00.153388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.153388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:00.197027 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:00.197824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:00.198737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:00.199139 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:00.199538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.200143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:00.200143 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.200143 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:00.200143 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.200143 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.200143 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:00.200143 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.200143 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.201771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:00.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:04.989492 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:04.992822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:05.026099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:05.026566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.027114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:05.132336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:05.133325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:05.133682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:05.133854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.134591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:05.134591 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.134591 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:05.134591 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.134591 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.134591 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:05.134591 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.134591 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:05.135468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.135768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:05.143604 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:05.143828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:05.144113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:05.144947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:05.145251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:05.145544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.146212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:05.146212 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.146212 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:05.146212 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.146212 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.146212 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:05.146212 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.146212 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:05.147172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:05.147567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:05.147567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.147567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:05.147567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.147567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.147567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:05.147567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.147567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:05.148353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.148799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.149288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:05.149288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.149288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:05.149288 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:05.149288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:05.151698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.152153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:05.152418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.153576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:05.197614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:05.198081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:05.198714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:05.199036 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:05.199203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.199761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:05.199761 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.199761 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:05.199761 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.199761 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.199761 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:05.199761 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.199761 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.200616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:05.201123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:09.991309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:09.992994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:10.023353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:10.023519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.023964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:10.023964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.023964 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:10.023964 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.023964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.023964 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:10.023964 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.023964 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.025124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.029001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:10.132924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:10.134869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:10.135197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:10.135441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.136315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:10.136315 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.136315 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:10.136315 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.136315 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.136315 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:10.136315 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.136315 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:10.138200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.138932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:10.144135 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:10.144453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:10.144897 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:10.145135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.145643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:10.145643 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.145643 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:10.145643 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.145643 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.145643 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:10.145643 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.145643 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:10.147389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:10.147888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:10.149512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.149961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.150449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:10.150449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.150449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:10.150449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:10.150564 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:10.150880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.151554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:10.151554 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.151554 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:10.151554 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.151554 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.151554 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:10.151554 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.151554 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:10.152428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:10.152926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:10.152926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.152926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:10.152926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.152926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.152926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:10.152926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.152926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:10.153751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.154376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:10.154819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.154819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:10.198620 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:10.199045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:10.199571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:10.200027 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:10.200265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.201164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:10.201164 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.201164 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:10.201164 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.201164 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.201164 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:10.201164 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.201164 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.202299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:10.203160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:14.991876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:14.996030 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:15.024637 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:15.024866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.025979 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:15.025979 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.025979 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:15.025979 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.025979 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.025979 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:15.025979 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.025979 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.026567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.027126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:15.134274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:15.135836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:15.136256 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:15.136455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.137379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:15.137379 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.137379 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:15.137379 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.137379 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.137379 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:15.137379 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.137379 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:15.138672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.139172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:15.144588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:15.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:15.145989 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:15.146339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.147152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:15.147152 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.147152 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:15.147152 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.147152 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.147152 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:15.147152 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.147152 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:15.148184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:15.148714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:15.148714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.148714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:15.148714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.148714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.148714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:15.148714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.148714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:15.149579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.150289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.151051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:15.151051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.151051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:15.151051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:15.151051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:15.151051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:15.151381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.154644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:15.154644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.154644 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:15.154644 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.154644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.154644 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:15.154644 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.154644 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:15.156091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:15.156570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:15.156570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.156570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:15.156570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.156570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.156570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:15.156570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.156570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:15.157796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.158319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:15.158792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.158792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:15.199161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:15.199670 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:15.200535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:15.200803 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:15.201024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.203383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:15.203383 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.203383 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:15.203383 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.203383 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.203383 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:15.203383 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.203383 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.206069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:15.206828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:19.992510 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:19.993523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:20.032016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:20.032345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.032767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:20.032767 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.032767 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:20.032767 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.032767 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.032767 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:20.032767 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.032767 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.033892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.034309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:20.134910 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:20.136100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:20.136247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:20.136373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.136966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:20.136966 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.136966 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:20.136966 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.136966 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.136966 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:20.136966 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.136966 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:20.138697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.139036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:20.145105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:20.145482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:20.145852 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:20.145967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.146473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:20.146473 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.146473 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:20.146473 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.146473 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.146473 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:20.146473 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.146473 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:20.147176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:20.147907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:20.147907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.147907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:20.147907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.147907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.147907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:20.147907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.147907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:20.148312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.149295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.150101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:20.150101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.150101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:20.150101 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:20.150603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:20.150918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:20.151094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.151508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:20.151508 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.151508 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:20.151508 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.151508 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.151508 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:20.151508 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.151508 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:20.152870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:20.153622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:20.153622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.153622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:20.153622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.153622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.153622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:20.153622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.153622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:20.154619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.155164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:20.155645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.155645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:20.199725 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:20.200770 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:20.200927 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:20.201251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:20.201400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.202251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:20.202251 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.202251 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:20.202251 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.202251 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.202251 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:20.202251 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.202251 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.203752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:20.204938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:24.993089 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:24.994078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:25.018065 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:25.018330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.018951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:25.018951 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.018951 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:25.018951 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.018951 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.018951 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:25.018951 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.018951 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.019827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.020576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:25.135879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:25.137119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:25.137347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:25.137471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.138300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:25.138300 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.138300 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:25.138300 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.138300 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.138300 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:25.138300 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.138300 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:25.140053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.140434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:25.145620 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:25.145944 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:25.146424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:25.146665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.147254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:25.147254 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.147254 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:25.147254 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.147254 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.147254 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:25.147254 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.147254 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:25.148137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:25.148647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:25.148647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.148647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:25.148647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.148647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.148647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:25.148647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.148647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:25.149397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.150250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.151550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:25.151550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.151550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:25.151550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:25.151550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:25.151550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:25.151761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.152056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:25.152056 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.152056 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:25.152056 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.152056 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.152056 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:25.152056 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.152056 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:25.153150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:25.153614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:25.153614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.153614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:25.153614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.153614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.153614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:25.153614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.153614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:25.154217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.155606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:25.156366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.156366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:25.200753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:25.201244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:25.201892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:25.202408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:25.202667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.203649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:25.203649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.203649 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:25.203649 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.203649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.203649 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:25.203649 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.203649 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.204758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:25.205505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:29.993555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:29.994825 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:29.996556 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=244 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:11:30.039465 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:30.039729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.040511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:30.040511 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.040511 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:30.040511 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.040511 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.040511 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:30.040511 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.040511 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.042056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.042632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:30.136497 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:30.137518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:30.137724 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:30.137880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.139219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:30.139219 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.139219 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:30.139219 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.139219 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.139219 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:30.139219 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.139219 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:30.140786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.142700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:30.146001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:30.146580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:30.147257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:30.147506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.148342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:30.148342 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.148342 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:30.148342 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.148342 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.148342 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:30.148342 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.148342 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:30.150236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:30.150676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:30.150676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.150676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:30.150676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.150676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.150676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:30.150676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.150676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:30.151449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.151861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.152383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:30.152383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.152383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:30.152383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:30.152383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:30.152866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:30.153371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.154466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:30.154466 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.154466 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:30.154466 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.154466 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.154466 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:30.154466 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.154466 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:30.155450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:30.155862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:30.155862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.155862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:30.155862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.155862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.155862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:30.155862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.155862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:30.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.157237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:30.157775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.157775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:30.201307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:30.201730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:30.202314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:30.202891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:30.203128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.204026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:30.204026 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.204026 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:30.204026 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.204026 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.204026 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:30.204026 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.204026 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.205205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:30.205569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:34.997425 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:34.999673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:35.024146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:35.024491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.025136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:35.025136 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.025136 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:35.025136 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.025136 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.025136 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:35.025136 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.025136 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.029082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.029629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:35.137109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:35.138009 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:35.138280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:35.138426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.139159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:35.139159 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.139159 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:35.139159 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.139159 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.139159 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:35.139159 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.139159 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:35.140320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.141152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:35.146415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:35.146777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:35.147291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:35.147576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.148283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:35.148283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.148283 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:35.148283 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.148283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.148283 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:35.148283 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.148283 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:35.148862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:35.149392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:35.149392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.149392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:35.149392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.149392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.149392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:35.149392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.149392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:35.149632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.150118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.150614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:35.150614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.150614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:35.152147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:35.152527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:35.154349 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:35.154349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:35.156795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.157406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:35.157805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.158326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:35.201750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:35.202290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:35.202874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:35.203287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:35.204019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.204912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:35.204912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.204912 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:35.204912 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.204912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.204912 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:35.204912 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.204912 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.206459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:35.207030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:40.025603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:40.026908 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:40.059309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:40.059577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.060717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:40.060717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.060717 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:40.060717 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.060717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.060717 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:40.060717 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.060717 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.062280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.062668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:40.137729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:40.138664 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:40.139037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:40.139297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.140227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:40.140227 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.140227 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:40.140227 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.140227 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.140227 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:40.140227 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.140227 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:40.142201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.142913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:40.146757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:40.147177 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:40.147506 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:40.147688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.148123 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:40.148123 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.148123 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:40.148123 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.148123 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.148123 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:40.148123 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.148123 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:40.148959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:40.149762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:40.149762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.149762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:40.149762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.149762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.149762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:40.149762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.149762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:40.150768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.151353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.151833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:40.151833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.151833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:40.152549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:40.153060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:40.153544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:40.154479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.154681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:40.154681 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.154681 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:40.154681 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.154681 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.154681 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:40.154681 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.154681 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:40.155563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:40.156028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:40.156028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.156028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:40.156028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.156028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.156028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:40.156028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.156028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:40.156622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.157130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:40.157641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.157641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:40.203096 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:40.203474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:40.204269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:40.204882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:40.204882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.205439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:40.205439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.205439 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:40.205439 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.205439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.205439 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:40.205439 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.205439 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.206525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:40.207322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:45.026131 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:45.028213 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:45.094792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:45.095072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.095691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Traceback (most recent call last):
Mar 26 01:11:45.095691 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.095691 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:45.095691 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.095691 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.095691 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:45.095691 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.095691 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-97fcc578-dc5a-4a49-b7bc-86658ba593bd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.096750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.097685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:45.138327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:45.139553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:45.139553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:45.139553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.140259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:45.140259 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.140259 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:45.140259 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.140259 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.140259 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:45.140259 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.140259 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:45.141040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.141701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:45.147270 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:45.147700 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:45.148189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:45.148457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.149566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:45.149566 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.149566 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:45.149566 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.149566 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.149566 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:45.149566 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.149566 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:45.150207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.151129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:45.151882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.152798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:45.153155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:45.153304 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:45.153604 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:45.153823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.154222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:45.154222 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.154222 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:45.154222 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.154222 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.154222 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:45.154222 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.154222 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:45.154914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:45.155348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:45.155348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.155348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:45.155348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.155348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.155348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:45.155348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.155348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:45.155958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.156348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:45.156766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.156766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:45.203699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:45.204181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:45.204783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:45.205143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:45.205369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.206039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:45.206039 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.206039 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:45.206039 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.206039 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.206039 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:45.206039 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.206039 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.207069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:45.207786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:50.026737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:50.027752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:50.093395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:50.093777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.094638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:11:50.094638 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.094638 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:50.094638 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.094638 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.094638 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:50.094638 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.094638 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.095556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.096111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:50.138861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:50.140159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:50.140159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:50.140159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.140591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:50.140591 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.140591 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:50.140591 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.140591 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.140591 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:50.140591 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.140591 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:50.141723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.142400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:50.147673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:50.148237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:50.148892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:50.149173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.149789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:50.149789 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.149789 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:50.149789 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.149789 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.149789 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:50.149789 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.149789 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:50.150841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.151969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:50.152843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:50.153859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:50.154634 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:50.154634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.155043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:50.155043 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.155043 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:50.155043 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.155043 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.155043 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:50.155043 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.155043 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:50.156052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:50.156812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:50.156812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.156812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:50.156812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.156812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.156812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:50.156812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.156812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:50.157238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.157689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:50.158193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.158193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:50.204373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:50.204961 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:50.205486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:50.205764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:50.205933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.206496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:50.206496 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.206496 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:50.206496 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.206496 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.206496 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:50.206496 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.206496 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.207577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:50.208471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:55.027405 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:55.028376 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:55.064711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:55.064996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.065847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:11:55.065847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.065847 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:55.065847 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.065847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.065847 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:55.065847 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.065847 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.066828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.067911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:55.139334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:55.140657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:55.140903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:55.141087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.141954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:11:55.141954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.141954 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:55.141954 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.141954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.141954 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:55.141954 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.141954 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:55.144407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.144872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:55.148162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:55.148774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:11:55.149174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:55.149363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.149883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:55.149883 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.149883 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:55.149883 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.149883 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.149883 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:55.149883 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.149883 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:55.151140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:55.151733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:55.151733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.151733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:55.151733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.151733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.151733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:55.151733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.151733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:11:55.152254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.152761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.153312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:55.153312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.153312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:55.153781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:55.154037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:55.154258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:55.154480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.154903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:55.154903 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.154903 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:55.154903 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.154903 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.154903 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:55.154903 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.154903 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:11:55.155724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:55.156089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:11:55.156089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.156089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:11:55.156089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.156089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.156089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:11:55.156089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.156089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:11:55.156756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.157094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:55.157452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.157452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:11:55.204936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:11:55.205532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:11:55.205949 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:55.206296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:11:55.206370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.206995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:11:55.206995 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.206995 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:11:55.206995 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.206995 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.206995 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:11:55.206995 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.206995 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.208784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:11:55.209462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:00.028003 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:00.029087 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:00.061947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:00.062267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.062989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:00.062989 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.062989 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:00.062989 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.062989 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.062989 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:00.062989 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.062989 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.064165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.064614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:00.140009 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:00.141133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:00.141758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:00.142026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.143170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:00.143170 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.143170 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:00.143170 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.143170 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.143170 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:00.143170 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.143170 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:00.144925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.146240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:00.150503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:00.150503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:00.150503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:00.150503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.152255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:00.152255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.152255 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:00.152255 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.152255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.152255 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:00.152255 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.152255 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:00.152838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:00.154060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:00.154060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.154060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:00.154060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.154060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.154060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:00.154060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.154060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:00.154864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.155297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.155683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:00.155683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.155683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:00.155683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:00.155683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:00.155800 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:00.156042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.157122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:00.157122 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.157122 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:00.157122 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.157122 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.157122 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:00.157122 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.157122 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:00.158278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:00.158596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:00.158596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.158596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:00.158596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.158596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.158596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:00.158596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.158596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:00.159820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.160148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:00.160494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.160494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:00.205806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:00.207567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:00.207941 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:00.208166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:00.208293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.209543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:00.210737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:05.029264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:05.030537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:05.071737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:05.072155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.072955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:05.072955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.072955 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:05.072955 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.072955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.072955 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:05.072955 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.072955 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.074486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.075227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:05.142136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:05.142136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:05.142136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:05.142444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.143364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:05.143364 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.143364 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:05.143364 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.143364 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.143364 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:05.143364 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.143364 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:05.144595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.145094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:05.149196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:05.149795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:05.150356 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:05.150642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.151402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:05.151402 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.151402 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:05.151402 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.151402 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.151402 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:05.151402 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.151402 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:05.152547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:05.152900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:05.152900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.152900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:05.152900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.152900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.152900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:05.152900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.152900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:05.153662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.154018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.154414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:05.154414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.154414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:05.155634 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:05.156353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:05.156734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:05.157015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.157593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:05.157593 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.157593 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:05.157593 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.157593 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.157593 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:05.157593 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.157593 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:05.158492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:05.158787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:05.158787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.158787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:05.158787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.158787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.158787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:05.158787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.158787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:05.160134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.160601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:05.161089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.161089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:05.209012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:05.209012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:05.209012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:05.209260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:05.209384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.210066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:05.210066 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.210066 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:05.210066 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.210066 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.210066 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:05.210066 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.210066 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.211823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:05.212232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:10.029399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:10.031394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:10.074576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:10.074774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.075427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:10.075427 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.075427 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:10.075427 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.075427 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.075427 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:10.075427 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.075427 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.076495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.077486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:10.141366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:10.142305 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:10.142580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:10.142675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.145352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:10.145352 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.145352 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:10.145352 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.145352 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.145352 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:10.145352 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.145352 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:10.146721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.147498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:10.149648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:10.149968 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:10.150403 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:10.150591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.151113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:10.151113 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.151113 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:10.151113 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.151113 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.151113 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:10.151113 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.151113 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:10.152301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:10.152760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:10.152760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.152760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:10.152760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.152760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.152760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:10.152760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.152760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:10.185286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.187570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.189092 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:10.190095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:10.190885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:10.191954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.192907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:10.208539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:10.209074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:10.209520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:10.209727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:10.209832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.210311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:10.210311 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.210311 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:10.210311 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.210311 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.210311 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:10.210311 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.210311 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:10.211968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:15.030158 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:15.036290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:15.068604 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.091939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:15.092740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:15.093445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.093445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:15.093445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.093445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.093445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:15.093445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.093445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:15.141886 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:15.142806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:15.143036 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:15.143203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.143583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:15.143583 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.143583 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:15.143583 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.143583 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.143583 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:15.143583 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.143583 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:15.144728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.145323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:15.150107 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:15.150413 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:15.150812 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:15.150923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.151342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:15.151342 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.151342 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:15.151342 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.151342 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.151342 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:15.151342 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.151342 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:15.152098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:15.152555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:15.152555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.152555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:15.152555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.152555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.152555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:15.152555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.152555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:15.153023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.153476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.153948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:15.153948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.153948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:15.156564 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:15.177330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:15.178102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:15.179143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:15.179939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:15.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:15.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:15.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:15.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:15.239654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.274199 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.274874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:15.275781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:20.030685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:20.031423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:20.083147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:20.083424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:20.086311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:20.087171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.087731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:20.087731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.087731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.087731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:20.087731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.087731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:20.142390 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:20.143381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:20.143600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:20.143657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.144277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:20.144277 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.144277 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:20.144277 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.144277 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.144277 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:20.144277 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.144277 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:20.145184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.146110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.177537 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:20.178220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.180074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:20.182592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:20.184570 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:20.185217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.185840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.188140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:20.189659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.190932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.190932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:20.190932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.190932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:20.240439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:20.240840 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:20.243092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:20.243413 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:20.243619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.244539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:20.244539 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.244539 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:20.244539 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.244539 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.244539 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:20.244539 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.244539 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.245746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:20.246559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:25.031498 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:25.033143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.068397 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.069563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.070213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:25.143088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:25.144652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:25.144652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:25.144652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.144957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:25.144957 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.144957 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:25.144957 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.144957 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.144957 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:25.144957 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.144957 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:25.146286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.146651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:25.150925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:25.151251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:25.151966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:25.151966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.152174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:25.152174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.152174 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:25.152174 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.152174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.152174 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:25.152174 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.152174 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:25.152879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:25.153355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:25.153355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.153355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:25.153355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.153355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.153355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:25.153355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.153355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:25.154312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.154668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.155066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:25.155066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.155066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:25.158088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:25.158539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:25.159259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:25.159259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.159414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:25.159414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.159414 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:25.159414 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.159414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.159414 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:25.159414 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.159414 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:25.160137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.160456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:25.160925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.161262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:25.161603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.161603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:25.240953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:25.241426 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:25.243507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:25.243507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:25.243507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.243704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:25.243704 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.243704 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:25.243704 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.243704 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.243704 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:25.243704 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.243704 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.245581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:25.246808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:30.032599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:30.033888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:30.036280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:12:30.075054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:30.076102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.077116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:30.077116 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.077116 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:30.077116 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.077116 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.077116 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:30.077116 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.077116 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.078321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.079687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:30.143932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:30.146803 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:30.148016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:30.148016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.149610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:30.149610 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.149610 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:30.149610 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.149610 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.149610 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:30.149610 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.149610 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:30.150507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.151725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:30.152408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:30.152408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:30.153767 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:30.153767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.154074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:12:30.154074 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.154074 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:30.154074 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.154074 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.154074 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:30.154074 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.154074 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:30.154887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.155423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:30.156094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.157067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.157724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:30.157724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.157724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:30.161237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:30.162303 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:30.163218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:30.163218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.163412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:30.163412 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.163412 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:30.163412 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.163412 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.163412 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:30.163412 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.163412 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:30.164187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:30.164929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:30.165625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.166569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:30.241431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:30.241833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:30.243764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:30.243764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:30.243764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.243871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:30.243871 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.243871 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:30.243871 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.243871 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.243871 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:30.243871 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.243871 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.244822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:30.246041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:35.033360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:35.034756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:35.074946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:35.075370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.076361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:35.076361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.076361 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:35.076361 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.076361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.076361 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:35.076361 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.076361 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.077664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.078241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:35.144773 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:35.146233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:35.146438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:35.146590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.148649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:35.148649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.148649 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:35.148649 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.148649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.148649 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:35.148649 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.148649 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:35.150013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.150482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.154015 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:35.154722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:35.155238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:35.155238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.155238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:35.155238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.155238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.155238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:35.155238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.155238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:35.155709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.156192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.156682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:35.156682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.156682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:35.162509 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:35.163234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:35.164186 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:35.164531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.165515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:35.165515 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.165515 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:35.165515 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.165515 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.165515 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:35.165515 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.165515 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:35.166911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.167370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:35.170068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.170618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:35.171226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.171226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:35.242001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:35.242264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:35.243082 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:35.243413 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:35.243590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.244326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:35.244326 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.244326 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:35.244326 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.244326 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.244326 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:35.244326 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.244326 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.245293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:35.245753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:40.033830 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:40.038219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:40.074110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:40.074439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.076432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:40.076432 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.076432 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:40.076432 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.076432 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.076432 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:40.076432 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.076432 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.076843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.077337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:40.145517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:40.146930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:40.147178 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:40.147257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.148112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:40.148112 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.148112 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:40.148112 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.148112 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.148112 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:40.148112 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.148112 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:40.149135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.150134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:40.152648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:40.152915 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:40.153242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:40.153424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.153770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:40.153770 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.153770 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:40.153770 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.153770 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.153770 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:40.153770 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.153770 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:40.154550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:40.155037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:40.155037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.155037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:40.155037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.155037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.155037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:40.155037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.155037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:40.155273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.156512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.157012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:40.157012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.157012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:40.163234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:40.163661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:40.164344 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:40.164634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.165283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:40.165283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.165283 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:40.165283 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.165283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.165283 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:40.165283 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.165283 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:40.166356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.166788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:40.167485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.168055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:40.168551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.168551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:40.242742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:40.243148 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:40.244016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:40.244353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:40.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.245145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:40.245145 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.245145 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:40.245145 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.245145 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.245145 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:40.245145 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.245145 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.246081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:40.246961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:45.038071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:45.038071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:45.077827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:45.078265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.079549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:45.079549 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.079549 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:45.079549 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.079549 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.079549 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:45.079549 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.079549 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.080577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.081789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:45.146171 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:45.147681 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:45.148086 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:45.148365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.149686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:45.149686 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.149686 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:45.149686 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.149686 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.149686 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:45.149686 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.149686 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:45.150865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.151309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:45.152995 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:45.153434 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:45.153985 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:45.154310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.155052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:45.155052 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.155052 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:45.155052 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.155052 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.155052 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:45.155052 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.155052 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:45.155945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:45.156392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:45.156392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.156392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:45.156392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.156392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.156392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:45.156392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.156392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:45.157496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.157927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.158458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:45.158458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.158458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:45.163611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:45.164195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:45.165195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:45.165605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.166463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:45.166463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.166463 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:45.166463 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.166463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.166463 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:45.166463 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.166463 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:45.167378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.167804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:45.168249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.168680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:45.169173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.169173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:45.244445 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:45.244917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:45.245894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:45.246306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:45.246474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.247329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:45.247329 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.247329 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:45.247329 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.247329 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.247329 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:45.247329 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.247329 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.248598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:45.249793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:50.037794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:50.038895 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:50.073584 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:50.073908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.074720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:50.074720 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.074720 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:50.074720 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.074720 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.074720 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:50.074720 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.074720 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.075915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.077257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:50.146820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:50.148345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:50.148672 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:50.148919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.149925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:50.149925 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.149925 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:50.149925 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.149925 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.149925 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:50.149925 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.149925 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:50.151420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.152351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:50.153429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:50.153788 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:50.154223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:50.154500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.155188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:50.155188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.155188 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:50.155188 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.155188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.155188 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:50.155188 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.155188 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:50.156256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:50.157099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:50.157099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.157099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:50.157099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.157099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.157099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:50.157099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.157099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:50.157784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.158638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.159241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:50.159241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.159241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:50.164212 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:50.164689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:50.165248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:50.165605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.166428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:50.166428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.166428 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:50.166428 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.166428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.166428 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:50.166428 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.166428 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:50.167071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.167608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:50.167845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.168346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:50.168856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.168856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:50.245410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:50.245645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:50.246382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:50.246861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:50.247199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.249208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:50.249208 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.249208 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:50.249208 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.249208 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.249208 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:50.249208 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.249208 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.250521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:50.251413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:55.038435 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:55.040058 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:55.072426 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:55.072796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.073691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:12:55.073691 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.073691 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:55.073691 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.073691 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.073691 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:55.073691 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.073691 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.074543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.075492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:55.147718 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:55.148883 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:55.149244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:55.149518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.150455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:12:55.150455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.150455 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:55.150455 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.150455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.150455 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:55.150455 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.150455 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:55.151706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.152497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:55.154019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:55.154221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:12:55.154988 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:55.155365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.156240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:55.156240 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.156240 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:55.156240 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.156240 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.156240 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:55.156240 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.156240 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:55.157542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:55.158074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:55.158074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.158074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:55.158074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.158074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.158074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:55.158074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.158074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:12:55.159156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.159620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.160141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:55.160141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.160141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:55.164665 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:55.165232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:55.165749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:55.166071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.166652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:12:55.166652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.166652 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:55.166652 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.166652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.166652 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:55.166652 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.166652 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:12:55.167569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.168001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:12:55.168628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.169135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:55.169700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.169700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:12:55.245729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:12:55.246132 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:12:55.246707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:55.247015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:12:55.247198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.247744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:12:55.247744 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.247744 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:12:55.247744 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.247744 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.247744 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:12:55.247744 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.247744 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.248591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:12:55.249757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:00.039613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:00.040521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:00.078493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:00.078842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.079690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:00.079690 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.079690 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:00.079690 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.079690 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.079690 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:00.079690 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.079690 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.080906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.081355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:00.148416 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:00.149621 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:00.149962 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:00.150217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.151336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:00.151336 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.151336 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:00.151336 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.151336 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.151336 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:00.151336 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.151336 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:00.152638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.153446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:00.154505 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:00.154788 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:00.155109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:00.155298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.155852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:00.155852 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.155852 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:00.155852 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.155852 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.155852 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:00.155852 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.155852 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:00.157408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:00.158340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:00.158340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.158340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:00.158340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.158340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.158340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:00.158340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.158340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:00.159106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.160740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:00.160740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.160740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:00.165575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:00.166059 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:00.166584 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:00.166853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.167602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:13:00.167602 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.167602 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:00.167602 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.167602 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.167602 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:00.167602 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.167602 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:00.168668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.169386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:00.170217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.170772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:00.171335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.171335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:00.246292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:00.246758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:00.247370 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:00.247764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:00.248001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.248881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:00.248881 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.248881 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:00.248881 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.248881 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.248881 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:00.248881 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.248881 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.250063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:00.251343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:05.040329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:05.041408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:05.070757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:05.071174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.072015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:05.072015 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.072015 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:05.072015 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.072015 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.072015 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:05.072015 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.072015 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.072959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.074388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:05.149045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:05.150199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:05.150822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:05.151216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.152233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:05.152233 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.152233 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:05.152233 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.152233 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.152233 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:05.152233 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.152233 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:05.153500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.154572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:05.155770 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:05.155770 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:05.155770 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:05.156003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.156792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:05.156792 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.156792 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:05.156792 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.156792 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.156792 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:05.156792 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.156792 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:05.157357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.158409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:05.159630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.162526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:05.166323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:05.167167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:05.168107 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:05.168107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.168842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:13:05.168842 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.168842 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:05.168842 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.168842 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.168842 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:05.168842 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.168842 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:05.170354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.170737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:05.172829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.173526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:05.174273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.174273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:05.246899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:05.247361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:05.247756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:05.248067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:05.248263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.248895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:05.248895 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.248895 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:05.248895 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.248895 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.248895 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:05.248895 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.248895 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.250811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:05.251166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:10.041162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:10.042520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:10.071504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:10.071893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.072900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:10.072900 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.072900 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:10.072900 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.072900 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.072900 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:10.072900 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.072900 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.074419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.075156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:10.149559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:10.150843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:10.151240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:10.151505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.152329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:10.152329 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.152329 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:10.152329 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.152329 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.152329 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:10.152329 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.152329 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:10.153478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.153932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:10.155408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:10.156004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:10.156353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:10.156549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.157075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:10.157075 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.157075 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:10.157075 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.157075 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.157075 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:10.157075 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.157075 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:10.157947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:10.158493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:10.158493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.158493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:10.158493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.158493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.158493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:10.158493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.158493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:10.159018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.159497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.160046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:10.160046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.160046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:10.166889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:10.167533 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:10.167963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:10.168200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.168788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:13:10.168788 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.168788 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:10.168788 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.168788 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.168788 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:10.168788 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.168788 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:10.170120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.170920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:10.171708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.172564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:10.173469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.173469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:10.247541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:10.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:10.248689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:10.249112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:10.249352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.250614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:10.250614 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.250614 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:10.250614 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.250614 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.250614 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:10.250614 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.250614 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.251963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:10.252858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:15.041760 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:15.043479 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:15.072517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:15.072770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.073478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:15.073478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.073478 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:15.073478 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.073478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.073478 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:15.073478 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.073478 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.074351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.074911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:15.150237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:15.151276 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:15.152145 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:15.152145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.152366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:15.152366 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.152366 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:15.152366 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.152366 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.152366 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:15.152366 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.152366 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:15.153452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.154176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:15.155869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:15.156062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:15.156230 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:15.156367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.156666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:15.156666 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.156666 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:15.156666 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.156666 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.156666 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:15.156666 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.156666 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:15.157194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.157835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:15.158476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.159305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:15.167401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:15.168061 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:15.168564 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:15.168934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.171310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:13:15.171310 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.171310 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:15.171310 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.171310 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.171310 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:15.171310 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.171310 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:15.172529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.173305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:15.174510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.175130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:15.175875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.175875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:15.248284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:15.248904 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:15.249337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:15.249692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:15.250125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.250756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:15.250756 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.250756 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:15.250756 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.250756 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.250756 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:15.250756 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.250756 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.251502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:15.251958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:20.042369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:20.043893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:20.079639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:20.079803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.080351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:20.080351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.080351 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:20.080351 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.080351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.080351 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:20.080351 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.080351 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.081230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.081643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:20.150692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:20.151835 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:20.152228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:20.152474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.153392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:20.153392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.153392 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:20.153392 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.153392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.153392 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:20.153392 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.153392 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:20.155094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.155681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:20.156257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:20.156721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:20.157819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:20.157819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.159133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:20.159133 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.159133 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:20.159133 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.159133 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.159133 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:20.159133 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.159133 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:20.159848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:20.160516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:20.160516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.160516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:20.160516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.160516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.160516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:20.160516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.160516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:20.161131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.162026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.162547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:20.162547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.162547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:20.167753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:20.168367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:20.168367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:20.168594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.169096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:13:20.169096 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.169096 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:20.169096 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.169096 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.169096 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:20.169096 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.169096 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:20.169671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:20.170640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:20.171711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.172800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:20.249235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:20.249738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:20.250394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:20.250863 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:20.251191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.252036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:20.252036 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.252036 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:20.252036 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.252036 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.252036 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:20.252036 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.252036 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.253854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:20.254313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:24.804933 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:24.813203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 26 01:13:24.813380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105681) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}}
Mar 26 01:13:24.814125 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Fixing resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:13:24.842628 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, resource={'id': 'f7e819e4-7101-412b-b047-baa5e16b2a52', 'name': 'admin_net', 'tenant_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'project_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f2e900ee-4d88-4151-8e50-661745950632'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:14Z', 'updated_at': '2026-03-26T01:02:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4803}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.842896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.843253 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d43d91f1-f24a-411f-bae7-af85048f95b9, external_ids_update=None, columns={'tag': 4803}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.843376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.844037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:13:24.844037 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:24.844037 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:24.844037 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:24.844037 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:24.844037 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:24.844037 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.844037 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Failed to fix resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:24.844990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:13:24.845305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:24.845565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:13:24.845565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.845565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:13:24.845565 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Fixing resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:13:24.871759 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-db438906-c74f-4dc8-883c-6e56962e551b, resource={'id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'name': 'public', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['1c3e4583-6e63-4953-8050-3341662a5ec6', '23682989-62bf-4e07-a74f-0cb01077d5ac'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-26T00:58:16Z', 'updated_at': '2026-03-26T00:58:25Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.872144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-db438906-c74f-4dc8-883c-6e56962e551b, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.872957 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-200618ec-0540-41b0-9429-dc7dfc74abb0, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.873318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.874044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:13:24.874044 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:24.874044 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:24.874044 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:24.874044 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:24.874044 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:24.874044 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.874044 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Failed to fix resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:24.874934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:13:24.875339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:24.875745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:13:24.875745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.875745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:13:24.875745 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Fixing resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:13:24.908660 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, resource={'id': 'f481c2f9-5052-46d5-821a-6c8712ec70ec', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['329ea1d3-f656-45b1-bf23-1d927d4578ea'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:50Z', 'updated_at': '2026-03-26T01:02:51Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19903}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.908995 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.911678 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3aa5be7a-5c67-4f1a-bdf2-7bf6912e871d, external_ids_update=None, columns={'tag': 19903}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.911829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.912730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:13:24.912730 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:24.912730 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:24.912730 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:24.912730 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:24.912730 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:24.912730 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.912730 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Failed to fix resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:24.914295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:13:24.914673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:24.915092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:13:24.915092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.915092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:13:24.915092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Fixing resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:13:24.942874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, resource={'id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'name': 'private', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['02d96a4b-4c93-4b95-ad0f-f11d085f0699', '0503b55a-1b69-49b5-b418-16ba37be90e9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T00:58:10Z', 'updated_at': '2026-03-26T00:58:13Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4449}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.943111 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.943345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6fe31530-ff9f-476b-aea1-4804059b76cd, external_ids_update=None, columns={'tag': 4449}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:24.943424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.943971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:13:24.943971 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:24.943971 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:24.943971 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:24.943971 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:24.943971 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:24.943971 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.943971 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Failed to fix resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:24.944917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:13:24.945342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:24.945736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:13:24.945736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:24.945736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:13:24.945736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Fixing resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:13:25.024412 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, resource={'id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'enable_snat': True}, 'gw_port_id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-ryt6n', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'ip_address': '172.24.5.142', 'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac'}, {'ip_address': '2001:db8::16a', 'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 3, 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8'}, resource_type=routers, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.024885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.025140 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.025355 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.025534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.026458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:13:25.026458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.026458 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:25.026458 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.026458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.026458 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:25.026458 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.026458 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:25.026686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Unable to update router e4efd798-3c25-4ea9-bd45-fa7a05aeaef9. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Failed to fix resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 26 01:13:25.028207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:13:25.028563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:13:25.028932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Fixing resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:13:25.043807 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:25.046461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:25.111183 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.111394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.112109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:25.112109 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.112109 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:25.112109 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.112109 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.112109 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:25.112109 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.112109 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.113776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.114408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:25.151701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:25.153284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.155702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.156079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.157043 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:25.157468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:25.157710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:25.157710 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.157710 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:25.157710 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.157710 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.157710 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:25.157710 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.157710 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:25.160415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.161613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:25.162382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.162382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.164401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:25.164401 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.164401 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:25.164401 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.164401 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.164401 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:25.164401 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.164401 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:25.166420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.167087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:25.167087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.167087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:25.167087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.167087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.167087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:25.167087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.167087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:25.169155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.169889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.170476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:25.170476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.170476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:25.170476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:25.171366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:25.176471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.176725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.179461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:13:25.179461 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.179461 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:25.179461 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.179461 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.179461 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:25.179461 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.179461 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:25.181296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.182048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:25.182894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.183428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:25.183928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.183928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:25.185803 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, resource={'id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'name': '', 'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'project_id': '', 'mac_address': 'fa:16:3e:4e:d4:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160849', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.186359 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::16a/64', '172.24.5.142/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.187067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=14a0aa42-40aa-4508-9af3-cffdffef9818, lrouter_port=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.187508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.188449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:13:25.188449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.188449 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:25.188449 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.188449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.188449 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:25.188449 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.188449 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Failed to fix resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:25.190416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.190667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.191047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:13:25.191047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.191047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:13:25.191047 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Maintenance task: Synchronization completed (took 0.38 seconds)
Mar 26 01:13:25.249804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:25.250524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:25.251248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.251611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:25.251855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.252629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:25.252629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.252629 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:25.252629 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.252629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.252629 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:25.252629 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.252629 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.254359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:25.254725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:30.044509 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:30.046323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:30.048346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=434 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:13:30.079574 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:30.079819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.080597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:30.080597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.080597 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:30.080597 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.080597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.080597 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:30.080597 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.080597 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.081829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.082714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:30.154734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:30.154734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:30.154734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:30.154734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.155147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:30.155147 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.155147 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:30.155147 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.155147 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.155147 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:30.155147 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.155147 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:30.156045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.156590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:30.157547 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:30.157857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:30.158157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:30.158382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.159929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:13:30.159929 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.159929 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:30.159929 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.159929 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.159929 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:30.159929 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.159929 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:30.160842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.161375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:30.161826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.162423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.162991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:30.162991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.162991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:30.170934 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:30.171395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:30.171684 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:30.171864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.172252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:30.172252 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.172252 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:30.172252 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.172252 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.172252 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:30.172252 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.172252 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:30.173233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:30.173730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:30.173730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.173730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:30.173730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.173730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.173730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:30.173730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.173730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:30.174227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.174913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:30.175664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.175664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:30.252012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:30.252012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:30.252012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:30.252012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:30.252012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.252398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:30.252398 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.252398 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:30.252398 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.252398 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.252398 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:30.252398 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.252398 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.253228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:30.253583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:35.045401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:35.046851 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:35.080461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:35.080776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.081481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:35.081481 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.081481 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:35.081481 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.081481 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.081481 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:35.081481 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.081481 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.082628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.083415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:35.153430 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:35.154491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:35.155076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:35.155562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.156558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:35.156558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.156558 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:35.156558 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.156558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.156558 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:35.156558 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.156558 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:35.158325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.158942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:35.159748 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:35.159748 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:35.160549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:35.160802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.161451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:13:35.161451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.161451 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:35.161451 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.161451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.161451 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:35.161451 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.161451 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:35.162473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.162833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:35.164061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.164497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.165038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:35.165038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.165038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:35.171346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:35.171901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:35.172367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:35.172652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.173308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:35.173308 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.173308 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:35.173308 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.173308 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.173308 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:35.173308 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.173308 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:35.176247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:35.177100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:35.177100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.177100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:35.177100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.177100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.177100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:35.177100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.177100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:35.178781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.179629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:35.180549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.180549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:35.251184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:35.251703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:35.252270 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:35.252856 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:35.253372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.254194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:35.254194 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.254194 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:35.254194 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.254194 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.254194 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:35.254194 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.254194 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.255764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:35.256595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:40.045789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:40.046711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:40.084309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:40.084561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.085222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:40.085222 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.085222 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:40.085222 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.085222 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.085222 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:40.085222 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.085222 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.086069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.086621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:40.154015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:40.154959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:40.155369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:40.155624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.156516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:40.156516 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.156516 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:40.156516 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.156516 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.156516 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:40.156516 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.156516 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:40.158459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.159334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:40.160214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:40.160366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:40.160886 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:40.161279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.161696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:13:40.161696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.161696 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:40.161696 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.161696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.161696 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:40.161696 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.161696 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:40.162335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.162810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:40.163254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.163737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.164250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:40.164250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.164250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:40.171602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:40.172139 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:40.172497 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:40.172779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.173519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:40.173519 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.173519 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:40.173519 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.173519 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.173519 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:40.173519 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.173519 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:40.174925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:40.175378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:40.175378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.175378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:40.175378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.175378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.175378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:40.175378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.175378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:40.176548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.177139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:40.177604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.177604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:40.251785 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:40.252360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:40.252901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:40.253310 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:40.253538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.254418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:40.254418 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.254418 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:40.254418 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.254418 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.254418 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:40.254418 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.254418 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.256004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:40.257019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:45.046696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:45.048888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:45.081991 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:45.082310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.083093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:45.083093 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.083093 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:45.083093 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.083093 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.083093 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:45.083093 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.083093 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.083947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.084572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:45.154682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:45.155729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:45.156040 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:45.156189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.157007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:45.157007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.157007 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:45.157007 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.157007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.157007 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:45.157007 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.157007 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:45.158785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.159203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:45.160023 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:45.160409 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:45.161328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:45.161770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.162968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:13:45.162968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.162968 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:45.162968 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.162968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.162968 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:45.162968 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.162968 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:45.163578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.163964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:45.164614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.165274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.166148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:45.166148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.166148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:45.172332 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:45.172761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:45.173448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:45.173855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.174582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:45.174582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.174582 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:45.174582 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.174582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.174582 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:45.174582 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.174582 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:45.175622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:45.176116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:45.176116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.176116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:45.176116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.176116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.176116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:45.176116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.176116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:45.176899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.177407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:45.177906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.177906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:45.252422 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:45.252996 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:45.253521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:45.254001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:45.254269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.255018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:45.255018 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.255018 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:45.255018 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.255018 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.255018 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:45.255018 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.255018 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.256587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:45.257401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:50.047546 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:50.048755 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:50.082705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:50.082929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.083637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:50.083637 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.083637 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:50.083637 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.083637 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.083637 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:50.083637 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.083637 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.084824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.085581 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:50.155342 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:50.156667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:50.157117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:50.157251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.158189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:50.158189 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.158189 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:50.158189 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.158189 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.158189 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:50.158189 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.158189 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:50.159879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.160626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:50.161410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:50.161410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:50.161753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:50.162118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.163226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:13:50.163226 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.163226 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:50.163226 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.163226 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.163226 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:50.163226 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.163226 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:50.164302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:50.165293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.166827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:50.172690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:50.173206 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:50.173496 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:50.173736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.174239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:50.174239 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.174239 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:50.174239 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.174239 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.174239 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:50.174239 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.174239 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:50.175168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:50.175647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:50.175647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.175647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:50.175647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.175647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.175647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:50.175647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.175647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:50.176302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.176813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:50.177394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.177394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:50.252943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:50.253623 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:50.254466 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:50.254835 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:50.255122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.255905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:50.255905 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.255905 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:50.255905 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.255905 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.255905 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:50.255905 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.255905 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.258228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:50.259030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:55.048272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:55.049295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:55.085687 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:55.086161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.086928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:13:55.086928 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.086928 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:55.086928 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.086928 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.086928 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:55.086928 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.086928 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.088098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.089169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:55.155943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:55.157553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:55.157919 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:55.158141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.158999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:13:55.158999 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.158999 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:55.158999 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.158999 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.158999 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:55.158999 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.158999 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:55.161012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.162051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:55.162569 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:55.162569 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:13:55.162689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:55.162890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.164262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:13:55.164262 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.164262 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:55.164262 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.164262 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.164262 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:55.164262 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.164262 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:55.165644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.166083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:13:55.167153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.167578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.168077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:55.168077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.168077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:55.173190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:55.173529 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:55.174115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:55.174319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.177149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:55.177149 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.177149 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:55.177149 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.177149 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.177149 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:55.177149 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.177149 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:13:55.177886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.178660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:55.179385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.180275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:13:55.253480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:13:55.254068 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:13:55.254692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:55.255093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:13:55.255313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.256089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:13:55.256089 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.256089 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:13:55.256089 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.256089 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.256089 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:13:55.256089 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.256089 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.257801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:13:55.258524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:00.048885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:00.050029 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:00.084918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:00.085251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.085950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:00.085950 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.085950 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:00.085950 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.085950 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.085950 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:00.085950 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.085950 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.087299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.088246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:00.156571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:00.157722 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:00.158480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:00.158480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.158940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:00.158940 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.158940 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:00.158940 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.158940 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.158940 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:00.158940 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.158940 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:00.160958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.161605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:00.162777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:00.162777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:00.162874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:00.163063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.163416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:00.163416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.163416 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:00.163416 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.163416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.163416 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:00.163416 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.163416 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:00.163946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.164443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:00.164697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.165156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.165657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:00.165657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.165657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:00.173736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:00.174137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:00.175062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:00.175281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.176034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:00.176034 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.176034 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:00.176034 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.176034 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.176034 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:00.176034 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.176034 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:00.176831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:00.177393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:00.177393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.177393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:00.177393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.177393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.177393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:00.177393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.177393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:00.178141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.178629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:00.179135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.179135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:00.254344 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:00.254903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:00.255370 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:00.255794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:00.256103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.256940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:00.256940 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.256940 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:00.256940 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.256940 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.256940 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:00.256940 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.256940 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.258692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:00.259329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:05.049415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:05.051105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:05.086085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:05.086270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.086994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:05.086994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.086994 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:05.086994 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.086994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.086994 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:05.086994 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.086994 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.088406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.088911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:05.157221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:05.157818 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:05.158024 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:05.158179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.158711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:05.158711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.158711 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:05.158711 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.158711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.158711 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:05.158711 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.158711 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:05.159651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.160600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:05.162762 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:05.163612 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:05.164024 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:05.164171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.164793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:05.164793 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.164793 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:05.164793 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.164793 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.164793 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:05.164793 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.164793 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:05.166902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.168300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:05.168895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.170299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.171038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:05.171038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.171038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:05.174040 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:05.174446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:05.174756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:05.174926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.175353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:05.175353 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.175353 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:05.175353 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.175353 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.175353 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:05.175353 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.175353 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:05.176223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:05.176850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:05.176850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.176850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:05.176850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.176850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.176850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:05.176850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.176850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:05.177221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:05.178557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.178557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:05.254913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:05.255385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:05.255787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:05.256100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:05.256258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.256839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:05.256839 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.256839 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:05.256839 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.256839 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.256839 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:05.256839 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.256839 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.258561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:05.258949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:10.050043 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:10.051510 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:10.082369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:10.082747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.083548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:10.083548 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.083548 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:10.083548 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.083548 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.083548 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:10.083548 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.083548 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.084484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.085346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:10.157735 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:10.158910 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:10.159337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:10.159771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.161351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:10.161351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.161351 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:10.161351 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.161351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.161351 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:10.161351 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.161351 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:10.163272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.163999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:10.164783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:10.164783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:10.164783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:10.164783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.165328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:10.165328 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.165328 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:10.165328 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.165328 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.165328 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:10.165328 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.165328 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:10.166587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.167051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:10.168313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.168769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.169282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:10.169282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.169282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:10.174506 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:10.175007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:10.175554 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:10.176017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.177137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:10.177137 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.177137 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:10.177137 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.177137 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.177137 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:10.177137 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.177137 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:10.178398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:10.179152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:10.179152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.179152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:10.179152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.179152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.179152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:10.179152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.179152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:10.180204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.180916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:10.181768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.181768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:10.255605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:10.256157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:10.256732 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:10.257229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:10.257434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.258006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:10.258006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.258006 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:10.258006 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.258006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.258006 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:10.258006 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.258006 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.259455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:10.260159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:15.050727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:15.052352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:15.088631 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:15.089173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.090137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:15.090137 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.090137 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:15.090137 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.090137 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.090137 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:15.090137 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.090137 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.091323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.092027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:15.158491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:15.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:15.160234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:15.160465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.161464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:15.161464 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.161464 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:15.161464 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.161464 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.161464 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:15.161464 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.161464 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:15.163323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.164349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:15.165501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:15.165501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:15.165501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:15.165501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.165885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:15.165885 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.165885 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:15.165885 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.165885 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.165885 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:15.165885 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.165885 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:15.166816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.167453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:15.167899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.168518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.169223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:15.169223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.169223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:15.174903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:15.175129 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:15.175629 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:15.176002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.176704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:15.176704 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.176704 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:15.176704 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.176704 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.176704 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:15.176704 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.176704 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:15.177436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:15.178009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:15.178009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.178009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:15.178009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.178009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.178009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:15.178009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.178009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:15.178534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.179211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:15.179850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.179850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:15.256167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:15.256796 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:15.257707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:15.258184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:15.258495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.259407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:15.259407 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.259407 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:15.259407 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.259407 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.259407 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:15.259407 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.259407 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.261248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:15.262771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:20.051453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:20.052629 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:20.088836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:20.089260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.090104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:20.090104 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.090104 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:20.090104 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.090104 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.090104 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:20.090104 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.090104 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.090925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.092264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:20.159159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:20.160332 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:20.160678 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:20.160953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.162150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:20.162150 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.162150 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:20.162150 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.162150 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.162150 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:20.162150 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.162150 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:20.163675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.164379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:20.164965 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:20.164965 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:20.165499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:20.165805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.166571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:20.166571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.166571 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:20.166571 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.166571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.166571 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:20.166571 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.166571 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:20.167796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.168328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:20.169356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.169834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.170384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:20.170384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.170384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:20.175193 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:20.175359 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:20.175852 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:20.176441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.177116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:20.177116 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.177116 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:20.177116 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.177116 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.177116 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:20.177116 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.177116 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:20.177920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:20.178399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:20.178399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.178399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:20.178399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.178399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.178399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:20.178399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.178399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:20.178881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.179473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:20.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:20.256802 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:20.257386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:20.258007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:20.258460 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:20.258721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.259638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:20.259638 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.259638 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:20.259638 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.259638 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.259638 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:20.259638 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.259638 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.261350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:20.262233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:25.052009 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:25.053031 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:25.084955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:25.085383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.086321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:25.086321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.086321 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:25.086321 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.086321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.086321 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:25.086321 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.086321 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.087461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.088372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:25.159866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:25.161180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:25.161632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:25.161743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.162668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:25.162668 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.162668 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:25.162668 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.162668 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.162668 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:25.162668 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.162668 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:25.164376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.165171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:25.165929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:25.165929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:25.165929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:25.166143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.166947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:25.166947 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.166947 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:25.166947 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.166947 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.166947 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:25.166947 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.166947 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:25.168424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.169227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:25.170222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.171036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.171549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:25.171549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.171549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:25.175455 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:25.175818 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:25.176091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:25.176424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.177227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:25.177227 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.177227 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:25.177227 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.177227 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.177227 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:25.177227 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.177227 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:25.177938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:25.178485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:25.178485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.178485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:25.178485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.178485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.178485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:25.178485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.178485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:25.178875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.179344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:25.179843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.179843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:25.257435 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:25.258008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:25.258636 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:25.259017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:25.259266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.260007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:25.260007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.260007 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:25.260007 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.260007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.260007 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:25.260007 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.260007 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.261672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:25.262529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:30.052522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:30.053539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:30.058439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=241 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:14:30.085220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:30.085565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.086692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:30.086692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.086692 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:30.086692 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.086692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.086692 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:30.086692 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.086692 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.087719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.088557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:30.160511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:30.161629 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:30.162004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:30.162341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.163633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:30.163633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.163633 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:30.163633 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.163633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.163633 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:30.163633 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.163633 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:30.164814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.165570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:30.166368 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:30.166368 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:30.166784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:30.167178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.168169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:30.168169 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.168169 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:30.168169 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.168169 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.168169 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:30.168169 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.168169 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:30.169513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.169968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:30.170996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.171513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.172066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:30.172066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.172066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:30.175744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:30.175931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:30.176259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:30.176492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.176999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:30.176999 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.176999 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:30.176999 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.176999 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.176999 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:30.176999 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.176999 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:30.177756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:30.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:30.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:30.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:30.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:30.178839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.179597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:30.180555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.180555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:30.258088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:30.258605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:30.259009 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:30.259305 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:30.259447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.260086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:30.260086 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.260086 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:30.260086 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.260086 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.260086 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:30.260086 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.260086 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.262259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:30.263066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:35.053318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:35.054916 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:35.087162 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:35.087530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.088439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:35.088439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.088439 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:35.088439 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.088439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.088439 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:35.088439 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.088439 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.089659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.090822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:35.161124 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:35.162093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:35.162431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:35.162693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.163823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:35.163823 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.163823 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:35.163823 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.163823 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.163823 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:35.163823 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.163823 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:35.165096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.166004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:35.166883 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:35.166883 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:35.167367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:35.167689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.168644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:35.168644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.168644 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:35.168644 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.168644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.168644 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:35.168644 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.168644 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:35.169841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.170333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:35.171600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.172112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.172644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:35.172644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.172644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:35.176270 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:35.176618 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:35.177066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:35.177298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.177761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:35.177761 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.177761 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:35.177761 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.177761 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.177761 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:35.177761 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.177761 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:35.179186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:35.179948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:35.179948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.179948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:35.179948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.179948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.179948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:35.179948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.179948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:35.180259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.180724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:35.181193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.181193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:35.258712 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:35.259166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:35.259802 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:35.260180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:35.260367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.260950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:35.260950 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.260950 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:35.260950 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.260950 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.260950 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:35.260950 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.260950 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.262889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:35.263766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:40.053841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:40.055293 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:40.086576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:40.086830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.087499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:40.087499 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.087499 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:40.087499 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.087499 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.087499 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:40.087499 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.087499 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.088661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.089591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:40.161515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:40.162572 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:40.163062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:40.163296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.164000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:40.164000 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.164000 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:40.164000 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.164000 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.164000 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:40.164000 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.164000 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:40.165064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.165958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:40.167103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:40.167103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:40.167914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:40.167914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.168320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:40.168320 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.168320 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:40.168320 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.168320 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.168320 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:40.168320 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.168320 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:40.169619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.170342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:40.171360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.172125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.173006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:40.173006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.173006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:40.176707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:40.177208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:40.177910 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:40.178369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.179050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:40.179050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.179050 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:40.179050 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.179050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.179050 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:40.179050 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.179050 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:40.180185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:40.180658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:40.180658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.180658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:40.180658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.180658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.180658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:40.180658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.180658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:40.181646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.182151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:40.182670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.182670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:40.259372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:40.259698 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:40.260309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:40.260731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:40.261143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.261901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:40.261901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.261901 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:40.261901 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.261901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.261901 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:40.261901 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.261901 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.263812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:40.264433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:45.054425 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:45.056164 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:45.089784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:45.090290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.091388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:45.091388 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.091388 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:45.091388 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.091388 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.091388 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:45.091388 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.091388 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.092527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.093034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:45.162012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:45.163351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:45.163792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:45.164168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.165204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:45.165204 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.165204 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:45.165204 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.165204 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.165204 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:45.165204 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.165204 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:45.166491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.167217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:45.168047 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:45.168047 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:45.168047 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:45.168047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.168248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:14:45.168248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.168248 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:45.168248 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.168248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.168248 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:45.168248 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.168248 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:45.169116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.169728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:45.170133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.170811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.171641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:45.171641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.171641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:45.177123 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:45.177424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:45.177868 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:45.178297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.178994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:45.178994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.178994 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:45.178994 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.178994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.178994 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:45.178994 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.178994 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:45.180153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:45.180568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:45.180568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.180568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:45.180568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.180568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.180568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:45.180568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.180568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:45.181793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.182268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:45.182752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.182752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:45.260375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:45.260375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:45.261202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:45.261415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:45.261482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.262163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:45.262163 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.262163 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:45.262163 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.262163 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.262163 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:45.262163 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.262163 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.263172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:45.264141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:50.054967 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:50.056922 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:50.090901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:50.091249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.092056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:50.092056 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.092056 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:50.092056 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.092056 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.092056 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:50.092056 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.092056 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.092924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.093801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:50.162579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:50.163726 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:50.164001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:50.164140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.164836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:50.164836 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.164836 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:50.164836 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.164836 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.164836 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:50.164836 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.164836 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:50.166690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.167476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:50.168278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:50.168278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:50.168278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:50.168278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.168762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:50.168762 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.168762 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:50.168762 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.168762 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.168762 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:50.168762 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.168762 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:50.169512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:50.170363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:50.170363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.170363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:50.170363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.170363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.170363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:50.170363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.170363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:50.170741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.171491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.172261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:50.172261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.172261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:50.177504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:50.177843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:50.178236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:50.178483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.178934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:14:50.178934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.178934 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:50.178934 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.178934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.178934 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:50.178934 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.178934 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:50.179478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.179896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:50.181555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.182803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:50.183819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.183819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:50.260528 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:50.261096 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:50.261707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:50.262112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:50.262249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.262828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:50.262828 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.262828 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:50.262828 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.262828 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.262828 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:50.262828 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.262828 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.264486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:50.265366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:55.055703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:55.057619 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:55.091156 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:55.091549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.092415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:14:55.092415 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.092415 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:55.092415 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.092415 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.092415 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:55.092415 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.092415 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.093535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.094184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:55.163234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:55.165021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:55.165362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:55.165602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.167101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:55.167101 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.167101 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:55.167101 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.167101 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.167101 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:55.167101 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.167101 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:55.169331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.170081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:55.170874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:55.170874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:14:55.170874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:55.170874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.171340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:14:55.171340 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.171340 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:55.171340 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.171340 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.171340 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:55.171340 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.171340 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:55.172231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.172907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:14:55.173344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.174154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.175092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:55.175092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.175092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:55.177870 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:55.178496 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:55.179775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:55.179775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.180033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:14:55.180033 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.180033 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:55.180033 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.180033 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.180033 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:55.180033 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.180033 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:14:55.180945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:55.182363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:55.183337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.184554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:14:55.261132 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:14:55.261638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:14:55.262173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:55.262549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:14:55.262780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.263572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:14:55.263572 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.263572 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:14:55.263572 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.263572 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.263572 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:14:55.263572 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.263572 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.265097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:14:55.265902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:00.056374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:00.057641 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:00.090505 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:00.090843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.091866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:00.091866 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.091866 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:00.091866 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.091866 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.091866 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:00.091866 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.091866 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.093145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.094101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:00.164657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:00.165915 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:00.166243 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:00.166423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.167174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:00.167174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.167174 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:00.167174 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.167174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.167174 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:00.167174 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.167174 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.168715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.169220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:00.169874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:00.170260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:00.170785 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:00.171081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.171644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:15:00.171644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.171644 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:00.171644 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.171644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.171644 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:00.171644 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.171644 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:00.172445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.172885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:00.173442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.173867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.174350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:00.174350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.174350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:00.178308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:00.179209 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:00.179841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:00.180196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:15:00.180944 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.180944 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:00.180944 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.180944 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.180944 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:00.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.180944 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:00.182533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.182930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:00.184353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.184886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:00.185395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.185395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:00.261778 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:00.262679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:00.263380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:00.263780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:00.264046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.265159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:00.265159 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.265159 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:00.265159 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.265159 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.265159 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:00.265159 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.265159 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.266692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:00.267669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:05.057308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:05.059355 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:05.093366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:05.093712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.094549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:05.094549 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.094549 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:05.094549 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.094549 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.094549 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:05.094549 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.094549 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.095424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.096370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:05.165313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:05.166333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:05.166601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:05.189779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:05.190339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:05.191088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:05.191645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:05.192135 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:05.192346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.192669 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:05.193071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:15:05.193071 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.193071 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:05.193071 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.193071 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.193071 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:05.193071 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.193071 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:05.193253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:05.194365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.194704 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:05.195622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.195964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.196350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:05.196350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.196350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:05.196416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:05.197131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.197496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:05.197944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.197944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:05.262324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:05.262727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:05.263613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:05.263999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:05.264375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.265519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:05.265519 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.265519 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:05.265519 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.265519 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.265519 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:05.265519 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.265519 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.267180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:05.268134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:10.058059 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:10.059930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:10.094581 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:10.095066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.097067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:10.097067 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.097067 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:10.097067 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.097067 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.097067 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:10.097067 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.097067 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.097963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.098638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:10.165942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:10.167398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:10.168038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:10.168038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.168964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:10.168964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.168964 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:10.168964 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.168964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.168964 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:10.168964 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.168964 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.170731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.171884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:10.191084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:10.191514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:10.192158 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:10.192520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:10.192963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:10.193190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.194161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:10.194556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:10.194556 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.194556 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:10.194556 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.194556 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.194556 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:10.194556 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.194556 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:10.195066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:10.195809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.196637 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:10.197477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.198288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:10.199306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:10.199865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.200291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.200808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:10.262813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:10.263724 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:10.264246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:10.264657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:10.264905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.265685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:10.265685 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.265685 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:10.265685 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.265685 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.265685 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:10.265685 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.265685 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.267459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:10.268398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:15.059424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:15.060822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:15.091733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:15.092228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.093188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:15.093188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.093188 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:15.093188 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.093188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.093188 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:15.093188 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.093188 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.093934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.095006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:15.166624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:15.168137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:15.168565 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:15.168932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.169768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:15.169768 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.169768 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:15.169768 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.169768 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.169768 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:15.169768 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.169768 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.172598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.173130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:15.191415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:15.191894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:15.192693 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:15.193228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:15.193994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.194408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:15.195222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:15.195222 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.195222 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:15.195222 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.195222 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.195222 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:15.195222 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.195222 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:15.196233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.196715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:15.197743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.198233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.198772 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:15.199715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.200203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:15.200830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.201321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:15.201837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.201837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:15.263784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:15.264388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:15.265152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:15.265472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:15.265695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.266533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:15.266533 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.266533 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:15.266533 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.266533 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.266533 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:15.266533 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.266533 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.268405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:15.269595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:20.060028 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:20.061411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:20.090458 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:20.090703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.091430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:20.091430 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.091430 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:20.091430 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.091430 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.091430 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:20.091430 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.091430 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.092602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.093195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:20.167172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:20.168024 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:20.168216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:20.168597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.168881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:20.168881 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.168881 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:20.168881 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.168881 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.168881 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:20.168881 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.168881 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.170097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.170443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:20.191897 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:20.192427 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:20.192714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:20.192880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.193339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:20.193339 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.193339 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:20.193339 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.193339 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.193339 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:20.193339 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.193339 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:20.194349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:20.195133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:20.195851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.196549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.197453 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:20.198240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:20.198664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:20.199436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.199945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:20.264229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:20.264809 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:20.265249 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:20.265469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:20.265613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.266325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:20.266325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.266325 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:20.266325 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.266325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.266325 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:20.266325 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.266325 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.268087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:20.268593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:25.060632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:25.062117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:25.096302 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:25.096458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.096877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:25.096877 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.096877 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:25.096877 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.096877 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.096877 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:25.096877 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.096877 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.097673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.098855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:25.167823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:25.168711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:25.169045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:25.169289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.170006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:25.170006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.170006 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:25.170006 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.170006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.170006 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:25.170006 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.170006 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.171791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.172900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:25.192440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:25.193226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:25.193892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:25.194195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.194946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:25.194946 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.194946 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:25.194946 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.194946 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.194946 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:25.194946 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.194946 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:25.196175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:25.196819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:25.197941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.198533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.199042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:25.199042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.199042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:25.199042 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:25.199575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:25.199921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.200758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Traceback (most recent call last):
Mar 26 01:15:25.200758 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.200758 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:25.200758 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.200758 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.200758 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:25.200758 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.200758 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:25.202081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.202637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1b773718-f388-43cf-ab13-eaf08f52afe5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:25.203606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.204067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:25.204567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.204567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:25.264792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:25.265218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:25.265731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:25.266129 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:25.266387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.267167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:25.267167 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.267167 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:25.267167 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.267167 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.267167 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:25.267167 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.267167 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.268207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:25.268962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:30.061354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:30.062296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:30.063685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=239 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:15:30.102269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:30.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.103054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:30.103054 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.103054 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:30.103054 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.103054 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.103054 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:30.103054 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.103054 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.104890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.105580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:30.168150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:30.168970 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:30.169139 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:30.169246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.169884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:30.169884 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.169884 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:30.169884 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.169884 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.169884 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:30.169884 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.169884 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.171826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.172674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:30.193362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:30.193898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:30.194492 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:30.194852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.195770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:30.195770 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.195770 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:30.195770 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.195770 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.195770 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:30.195770 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.195770 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:30.197247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.197946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:30.199444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.200406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.201251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:30.201251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.201251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:30.201251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:30.201251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:30.201251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:30.201682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.202230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:30.202230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.202230 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:30.202230 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.202230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.202230 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:30.202230 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.202230 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:30.203492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.203911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:30.204911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.205359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:30.205847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.205847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:30.265486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:30.265908 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:30.266559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:30.266936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:30.267230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.268088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:30.268088 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.268088 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:30.268088 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.268088 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.268088 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:30.268088 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.268088 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.269538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:30.270315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:35.062010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:35.063651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:35.096227 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:35.096660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.097566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:35.097566 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.097566 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:35.097566 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.097566 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.097566 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:35.097566 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.097566 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.098758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.099405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:35.168706 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:35.169914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:35.170236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:35.170466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.171241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:35.171241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.171241 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:35.171241 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.171241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.171241 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:35.171241 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.171241 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.173223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.173760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:35.193833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:35.194289 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:35.194745 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:35.195040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.195817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:35.195817 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.195817 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:35.195817 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.195817 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.195817 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:35.195817 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.195817 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:35.197180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.197645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:35.198877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.199381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.200034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:35.200034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.200034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:35.200266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:35.200847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:35.201372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:35.201725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.202525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:15:35.202525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.202525 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:35.202525 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.202525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.202525 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:35.202525 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.202525 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:35.203813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.204217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:35.205489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.205942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:35.206455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.206455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:35.266349 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:35.266888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:35.267451 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:35.267817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:35.268098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.269096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:35.269096 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.269096 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:35.269096 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.269096 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.269096 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:35.269096 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.269096 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.270663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:35.271211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:40.062682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:40.064199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:40.101825 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:40.102298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.103268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:40.103268 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.103268 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:40.103268 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.103268 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.103268 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:40.103268 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.103268 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.104473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.105380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:40.169450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:40.170663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:40.171011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:40.171226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.172114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:40.172114 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.172114 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:40.172114 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.172114 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.172114 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:40.172114 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.172114 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.173454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.174006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:40.194563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:40.195035 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:40.195541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:40.195830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.196505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:40.196505 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.196505 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:40.196505 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.196505 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.196505 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:40.196505 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.196505 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:40.197067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.197535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:40.197827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.198329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.198856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:40.198856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.198856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:40.200714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:40.201292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:40.201858 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:40.202221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.202969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:15:40.202969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.202969 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:40.202969 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.202969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.202969 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:40.202969 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.202969 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:40.204475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.204931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:40.206297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.206773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:40.207350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.207350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:40.266951 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:40.267543 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:40.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:40.268710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:40.269066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.270052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:40.270052 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.270052 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:40.270052 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.270052 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.270052 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:40.270052 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.270052 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.271932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:40.272848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:45.063253 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:45.064772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:45.096057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:45.096736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.097599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:45.097599 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.097599 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:45.097599 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.097599 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.097599 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:45.097599 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.097599 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.098582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.099689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:45.170121 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:45.171384 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:45.171676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:45.171900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.172740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:45.172740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.172740 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:45.172740 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.172740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.172740 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:45.172740 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.172740 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.174600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.175876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:45.195208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:45.195757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:45.196501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:45.196800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.197604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:45.197604 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.197604 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:45.197604 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.197604 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.197604 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:45.197604 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.197604 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:45.198911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.199738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:45.200781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.201559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.202450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:45.202450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.202450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:45.202450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:45.202450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:45.202450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:45.202450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.203052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:15:45.203052 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.203052 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:45.203052 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.203052 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.203052 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:45.203052 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.203052 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:45.203926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.204710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:45.205484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.206096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:45.206574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.206574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:45.267666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:45.268060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:45.268607 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:45.268873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:45.269101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.269573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:45.269573 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.269573 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:45.269573 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.269573 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.269573 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:45.269573 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.269573 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.270565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:45.271358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:50.063867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:50.064813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:50.091835 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:50.092293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.092962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:50.092962 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.092962 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:50.092962 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.092962 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.092962 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:50.092962 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.092962 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.093771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.094852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:50.170931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:50.171770 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:50.172076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:50.172368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.173319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:50.173319 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.173319 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:50.173319 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.173319 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.173319 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:50.173319 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.173319 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.174556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.175331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:50.195763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:50.196354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:50.197041 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:50.197316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.198110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:50.198110 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.198110 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:50.198110 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.198110 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.198110 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:50.198110 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.198110 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:50.199084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.199405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:50.200472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.200801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.201178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:50.201178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.201178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:50.201518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:50.201838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:50.202535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:50.202727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.203127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:15:50.203127 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.203127 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:50.203127 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.203127 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.203127 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:50.203127 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.203127 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:50.204090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.204371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:50.205720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.206063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:50.206430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.206430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:50.268210 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:50.268686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:50.269239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:50.269653 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:50.269875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.270583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:50.270583 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.270583 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:50.270583 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.270583 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.270583 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:50.270583 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.270583 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.272046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:50.272405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:55.064290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:55.065793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:55.100107 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:55.100306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.100910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:15:55.100910 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.100910 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:55.100910 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.100910 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.100910 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:55.100910 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.100910 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.102523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.103286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:55.171155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:55.172325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:55.172709 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:55.172954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.173870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:55.173870 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.173870 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:55.173870 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.173870 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.173870 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:55.173870 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.173870 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.175812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.176152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:55.196296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:55.196713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:15:55.197303 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:55.197583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.198381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:15:55.198381 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.198381 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:55.198381 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.198381 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.198381 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:55.198381 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.198381 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:55.199657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.200541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:15:55.201356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.202232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.202765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:55.202765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.202765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:55.202765 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:55.202765 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:55.203057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:55.203395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.204179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:15:55.204179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.204179 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:55.204179 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.204179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.204179 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:55.204179 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.204179 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:15:55.205626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.206105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:15:55.207459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.207918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:55.208546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.208546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:15:55.268725 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:15:55.269115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:15:55.269585 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:55.269937 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:15:55.270200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.270845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:15:55.270845 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.270845 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:15:55.270845 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.270845 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.270845 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:15:55.270845 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.270845 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.272255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:15:55.273204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:00.064962 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:00.066794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:00.099201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:00.099400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.100102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:00.100102 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.100102 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:00.100102 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.100102 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.100102 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:00.100102 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.100102 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.100987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.101628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:00.171618 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:00.173170 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:00.173469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:00.173683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.174470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:00.174470 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.174470 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:00.174470 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.174470 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.174470 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:00.174470 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.174470 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.176529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.177757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:00.196871 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:00.199882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:00.200526 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:00.200813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.201689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:16:00.201689 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.201689 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:00.201689 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.201689 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.201689 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:00.201689 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.201689 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:00.203349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:00.204170 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:00.206156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.206767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.207279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:00.207279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.207279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:00.207279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:00.207279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:00.207881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.208743 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:00.210258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.210842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:00.212269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.212740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:00.213239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.213239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:00.269239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:00.269761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:00.270330 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:00.270603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:00.270758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.271438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:00.271438 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.271438 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:00.271438 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.271438 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.271438 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:00.271438 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.271438 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.272593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:00.273198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:05.066220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:05.067106 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:05.099436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:05.099704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.100384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:05.100384 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.100384 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:05.100384 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.100384 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.100384 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:05.100384 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.100384 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.101542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.102473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:05.172172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:05.173591 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:05.174128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:05.174908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.175590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:05.175590 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.175590 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:05.175590 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.175590 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.175590 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:05.175590 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.175590 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.177535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.178261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:05.197384 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:05.197944 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:05.198406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:05.198718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.199875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:16:05.199875 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.199875 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:05.199875 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.199875 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.199875 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:05.199875 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.199875 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:05.200531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.201505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:05.201953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.202844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.203585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:05.203585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.203585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:05.204973 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:05.205466 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:05.206243 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:05.206425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.207090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:05.207090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.207090 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:05.207090 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.207090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.207090 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:05.207090 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.207090 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:05.208266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.208826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:05.209884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.210322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:05.210821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.210821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:05.269899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:05.270343 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:05.270917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:05.271213 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:05.271334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.271912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:05.271912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.271912 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:05.271912 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.271912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.271912 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:05.271912 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.271912 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.273593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:05.274565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:10.066517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:10.067954 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:10.100255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:10.100538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.101455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:10.101455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.101455 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:10.101455 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.101455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.101455 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:10.101455 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.101455 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.102421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.103358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:10.172679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:10.173848 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:10.174045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:10.174200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.175463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:10.175463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.175463 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:10.175463 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.175463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.175463 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:10.175463 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.175463 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.178038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.178844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:10.198007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:10.198327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:10.198753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:10.198954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.199545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:16:10.199545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.199545 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:10.199545 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.199545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.199545 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:10.199545 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.199545 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:10.200672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.201434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:10.201850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.202596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.203419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:10.203419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.203419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:10.205319 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:10.205595 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:10.205893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:10.206139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.206593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:10.206593 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.206593 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:10.206593 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.206593 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.206593 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:10.206593 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.206593 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:10.207330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.207876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:10.208352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.208791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:10.209355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.209355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:10.270482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:10.270956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:10.271695 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:10.271890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:10.272005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.272504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:10.272504 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.272504 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:10.272504 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.272504 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.272504 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:10.272504 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.272504 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.273657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:10.274589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:15.067146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:15.068705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:15.105005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:15.105499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.106419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:15.106419 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.106419 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:15.106419 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.106419 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.106419 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:15.106419 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.106419 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.107528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.108448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:15.173338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:15.174627 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:15.175166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:15.175391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.176355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:15.176355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.176355 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:15.176355 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.176355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.176355 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:15.176355 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.176355 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.178786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.179578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:15.198467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:15.198875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:15.199499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:15.199893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.200777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:16:15.200777 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.200777 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:15.200777 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.200777 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.200777 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:15.200777 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.200777 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:15.202105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.202749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:15.203657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.204579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.205494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:15.205494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.205494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:15.205630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:15.205824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:15.206413 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:15.206689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.207224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:15.207224 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.207224 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:15.207224 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.207224 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.207224 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:15.207224 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.207224 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:15.207948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.208365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:15.208924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.209364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:15.209874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.209874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:15.271142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:15.271577 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:15.272233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:15.272632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:15.272877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.273709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:15.273709 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.273709 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:15.273709 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.273709 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.273709 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:15.273709 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.273709 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.275501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:15.276396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:20.067877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:20.069423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:20.102823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:20.103178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.103876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:20.103876 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.103876 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:20.103876 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.103876 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.103876 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:20.103876 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.103876 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.104864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.105742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:20.173859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:20.198905 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:20.201286 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:20.202085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:20.202464 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:20.202711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.203695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:16:20.203695 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.203695 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:20.203695 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.203695 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.203695 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:20.203695 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.203695 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:20.204365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:20.206595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.207074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:20.207613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.207654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:20.208130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:16:20.208130 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.208130 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:20.208130 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.208130 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.208130 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:20.208130 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.208130 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:20.210065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.210569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:20.211716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.212246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.212803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:20.212803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.212803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:20.212803 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:20.212803 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:20.212999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.213421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:20.213421 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.213421 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:20.213421 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.213421 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.213421 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:20.213421 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.213421 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:20.214544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.215026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:20.215608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.216135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:20.216683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.216683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:20.271767 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:20.272110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:20.272956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:20.273442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:20.273708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.274606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:20.274606 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.274606 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:20.274606 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.274606 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.274606 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:20.274606 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.274606 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.276261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:20.276762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:25.068566 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:25.070365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:25.105936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:25.106247 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.106905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:25.106905 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.106905 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:25.106905 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.106905 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.106905 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:25.106905 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.106905 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.107805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.108722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:25.200688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:25.201568 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:25.202116 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:25.202298 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:25.202445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.202947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:25.203539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:25.203539 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.203539 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:25.203539 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.203539 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.203539 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:25.203539 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.203539 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.204629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.205379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:25.206138 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:25.206138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.206488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:16:25.206488 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.206488 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:25.206488 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.206488 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.206488 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:25.206488 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.206488 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:25.207993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.208753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:25.210036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.210574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.211086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:25.211086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.211086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:25.211169 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:25.211910 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:25.212522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:25.212803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.213494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:16:25.213494 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.213494 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:25.213494 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.213494 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.213494 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:25.213494 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.213494 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:25.215080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.215463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:25.217021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.217886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:25.218718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.218718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:25.272688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:25.273232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:25.273835 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:25.274153 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:25.274312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.274938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:25.274938 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.274938 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:25.274938 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.274938 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.274938 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:25.274938 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.274938 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.276650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:25.277209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:30.069632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:30.071380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:30.073078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:16:30.108246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:30.108676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.109668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:30.109668 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.109668 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:30.109668 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.109668 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.109668 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:30.109668 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.109668 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.110802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.111813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:30.201507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:30.202753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:30.203034 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:30.203239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.203919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:30.203919 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.203919 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:30.203919 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.203919 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.203919 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:30.203919 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.203919 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:30.204525 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.206445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.207467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.208366 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:30.209019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.209860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:30.210532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.211588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.212614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:30.212614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.212614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:30.212614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:30.212614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:30.212614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:30.212814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.213196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:30.213196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.213196 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:30.213196 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.213196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.213196 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:30.213196 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.213196 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:30.233206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:30.233734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:30.234144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.234623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:30.273307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:30.274539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:30.275180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:30.275689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:30.275996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.276935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:30.276935 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.276935 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:30.276935 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.276935 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.276935 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:30.276935 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.276935 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.278668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:30.279280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:35.070445 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:35.072863 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:35.109541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:35.110121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.110876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:35.110876 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.110876 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:35.110876 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.110876 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.110876 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:35.110876 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.110876 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.111891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.112633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:35.202136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:35.203291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:35.203603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:35.203839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.204595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:35.204595 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.204595 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:35.204595 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.204595 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.204595 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:35.204595 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.204595 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.206816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.207319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:35.207819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:35.207819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:35.208195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:35.208567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.209570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:35.209570 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.209570 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:35.209570 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.209570 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.209570 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:35.209570 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.209570 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:35.210379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:35.210868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:35.210868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.210868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:35.210868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.210868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.210868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:35.210868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.210868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:35.211299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.211770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.212311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:35.212311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.212311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:35.212311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:35.212669 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:35.213105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:35.213353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.213946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:16:35.213946 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.213946 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:35.213946 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.213946 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.213946 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:35.213946 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.213946 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:35.215451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.215896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:35.217404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.217861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:35.218408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.218408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:35.274630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:35.275238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:35.275828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:35.276284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:35.276523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.277368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:35.277368 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.277368 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:35.277368 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.277368 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.277368 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:35.277368 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.277368 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.279194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:35.279948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:40.071182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:40.072489 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:40.103896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:40.104220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.104903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:40.104903 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.104903 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:40.104903 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.104903 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.104903 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:40.104903 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.104903 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.106135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.106689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:40.202789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:40.204199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:40.204532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:40.204755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.205677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:40.205677 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.205677 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:40.205677 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.205677 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.205677 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:40.205677 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.205677 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.207481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.208278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:40.209054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:40.209054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:40.209054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:40.209295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.210070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:16:40.210070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.210070 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:40.210070 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.210070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.210070 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:40.210070 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.210070 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:40.211480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.212254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:40.213096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.213803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.214352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:40.214352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.214352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:40.214352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:40.214352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:40.214524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:40.214791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.215554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:40.215554 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.215554 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:40.215554 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.215554 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.215554 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:40.215554 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.215554 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:40.216503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.217000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:40.217664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.218163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:40.218709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.218709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:40.275326 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:40.276207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:40.276368 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:40.276662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:40.276892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.277467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:40.277467 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.277467 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:40.277467 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.277467 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.277467 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:40.277467 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.277467 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.278355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:40.279144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:45.071709 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:45.073279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:45.103923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:45.104347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.105169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:45.105169 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.105169 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:45.105169 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.105169 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.105169 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:45.105169 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.105169 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.105994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.106859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:45.203480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:45.204661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:45.205015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:45.205281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.206052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:45.206052 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.206052 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:45.206052 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.206052 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.206052 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:45.206052 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.206052 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.207749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.208588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:45.209526 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:45.209526 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:45.209526 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:45.209819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.210664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:16:45.210664 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.210664 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:45.210664 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.210664 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.210664 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:45.210664 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.210664 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:45.211846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.212336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:45.213691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.214324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.214806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:45.214806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.214806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:45.214806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:45.214806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:45.215069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:45.215069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.215557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:45.215557 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.215557 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:45.215557 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.215557 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.215557 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:45.215557 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.215557 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:45.216516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.216953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:45.217642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.218108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:45.218579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.218579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:45.276013 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:45.276508 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:45.277148 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:45.277703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:45.277951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.278657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:45.278657 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.278657 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:45.278657 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.278657 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.278657 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:45.278657 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.278657 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.280326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:45.281559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:50.072375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:50.074032 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:50.109093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:50.109471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.110374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:50.110374 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.110374 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:50.110374 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.110374 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.110374 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:50.110374 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.110374 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.111568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.112956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:50.204262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:50.205372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:50.205696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:50.205935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.206794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:50.206794 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.206794 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:50.206794 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.206794 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.206794 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:50.206794 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.206794 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.208500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.209272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:50.210067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:50.210067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:50.210067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:50.210067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.210934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:16:50.210934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.210934 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:50.210934 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.210934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.210934 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:50.210934 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.210934 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:50.212312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.212761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:50.213865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.214330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.214836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:50.214836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.214836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:50.214836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:50.215091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:50.215601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:50.215931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.216696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:50.216696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.216696 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:50.216696 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.216696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.216696 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:50.216696 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.216696 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:50.218013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.218441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:50.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.220029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:50.220501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.220501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:50.276493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:50.276812 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:50.277278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:50.277609 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:50.277815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.278506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:50.278506 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.278506 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:50.278506 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.278506 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.278506 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:50.278506 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.278506 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.279500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:50.280687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:55.072937 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:55.074364 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:55.106410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:55.106808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.107706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:16:55.107706 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.107706 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:55.107706 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.107706 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.107706 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:55.107706 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.107706 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.108946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.109643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:55.204802 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:55.206149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:55.206540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:55.206805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.207713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:55.207713 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.207713 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:55.207713 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.207713 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.207713 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:55.207713 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.207713 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.209490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.210256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:55.211010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:55.211010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:16:55.211010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:55.211368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.211817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:16:55.211817 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.211817 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:55.211817 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.211817 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.211817 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:55.211817 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.211817 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:55.213142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.213932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:16:55.214666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.215455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.216330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:55.216330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.216330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:55.216330 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:55.216553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:55.217073 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:55.217299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.218007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:16:55.218007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.218007 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:55.218007 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.218007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.218007 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:55.218007 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.218007 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:16:55.219394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.219797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:16:55.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.222221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:55.222721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.222721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:16:55.277028 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:16:55.277505 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:16:55.278163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:55.278640 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:16:55.279005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.279904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:16:55.279904 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.279904 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:16:55.279904 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.279904 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.279904 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:16:55.279904 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.279904 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.281473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:16:55.282304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:00.073576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:00.074765 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:00.109449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:00.109747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.110476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:00.110476 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.110476 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:00.110476 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.110476 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.110476 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:00.110476 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.110476 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.111694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.112589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:00.205469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:00.206640 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:00.206945 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:00.207224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.208112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:00.208112 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.208112 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:00.208112 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.208112 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.208112 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:00.208112 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.208112 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.209319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.210168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:00.210894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:00.210894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:00.211059 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:00.211325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.211996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:17:00.211996 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.211996 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:00.211996 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.211996 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.211996 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:00.211996 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.211996 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:00.212799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.213571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:00.214207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.214894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.215719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:00.215719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.215719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:00.215719 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:00.216223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:00.216743 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:00.217130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.217863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:17:00.217863 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.217863 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:00.217863 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.217863 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.217863 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:00.217863 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.217863 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:00.219150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.219619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:00.220752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.221275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:00.221816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.221816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:00.277534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:00.278136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:00.278662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:00.279041 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:00.279258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.279923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:00.279923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.279923 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:00.279923 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.279923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.279923 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:00.279923 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.279923 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.281545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:00.282122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:05.074265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:05.075953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:05.115857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:05.116297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.117154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:05.117154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.117154 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:05.117154 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.117154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.117154 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:05.117154 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.117154 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.118194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.119079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:05.206158 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:05.207442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:05.207795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:05.208070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.209333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:05.209333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.209333 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:05.209333 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.209333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.209333 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:05.209333 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.209333 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.211204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.211917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:05.212647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:05.212647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:05.212647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:05.212822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.213633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:17:05.213633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.213633 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:05.213633 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.213633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.213633 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:05.213633 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.213633 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:05.214938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.215659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:05.216302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.216915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.217563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:05.217563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.217563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:05.217563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:05.217563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:05.217563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:05.217831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.218175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:17:05.218175 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.218175 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:05.218175 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.218175 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.218175 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:05.218175 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.218175 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:05.219210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.219850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:05.220645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:05.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.221764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:05.278157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:05.278530 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:05.278997 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:05.279511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:05.279812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.280467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:05.280467 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.280467 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:05.280467 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.280467 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.280467 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:05.280467 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.280467 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.281960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:05.282735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:10.074934 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:10.076117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:10.110601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:10.111003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.111841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:10.111841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.111841 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:10.111841 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.111841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.111841 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:10.111841 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.111841 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.113166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.114118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:10.206786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:10.208334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:10.208647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:10.208870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.209776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:10.209776 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.209776 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:10.209776 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.209776 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.209776 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:10.209776 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.209776 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.211546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.212351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:10.213155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:10.213155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:10.213155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:10.213155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.213871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:17:10.213871 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.213871 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:10.213871 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.213871 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.213871 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:10.213871 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.213871 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:10.214659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.215428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:10.215874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.216654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.217518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:10.217518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.217518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:10.217518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:10.218350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:10.218687 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:10.218898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.219586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:17:10.219586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.219586 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:10.219586 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.219586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.219586 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:10.219586 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.219586 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:10.220877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.221614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:10.222838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.223641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:10.224507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.224507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:10.278664 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:10.279115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:10.279752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:10.280204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:10.280420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.281166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:10.281166 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.281166 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:10.281166 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.281166 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.281166 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:10.281166 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.281166 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.282732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:10.283568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:15.075652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:15.077361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:15.114617 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:15.114846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.115217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:15.115217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.115217 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:15.115217 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.115217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.115217 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:15.115217 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.115217 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.116067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.117108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:15.207619 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:15.208788 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:15.209153 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:15.209391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.210223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:15.210223 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.210223 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:15.210223 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.210223 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.210223 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:15.210223 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.210223 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.211388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.211921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:15.212457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:15.212571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:15.213062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:15.213355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.214132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:17:15.214132 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.214132 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:15.214132 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.214132 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.214132 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:15.214132 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.214132 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:15.214903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.215675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:15.216196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.216913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.217761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:15.217761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.217761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:15.218106 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:15.218720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:15.219235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:15.219452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.220065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:17:15.220065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.220065 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:15.220065 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.220065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.220065 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:15.220065 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.220065 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:15.221363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.221798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:15.223139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.223615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:15.224153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.224153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:15.279296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:15.279601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:15.280193 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:15.280644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:15.280882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.281739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:15.281739 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.281739 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:15.281739 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.281739 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.281739 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:15.281739 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.281739 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.282869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:15.283411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:20.076350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:20.077936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:20.114473 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:20.114675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.115278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:20.115278 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.115278 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:20.115278 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.115278 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.115278 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:20.115278 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.115278 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.116375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.117157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:20.208176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:20.209150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:20.209470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:20.209700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.210510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:20.210510 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.210510 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:20.210510 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.210510 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.210510 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:20.210510 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.210510 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.211376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.212181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:20.213133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:20.213503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:20.213992 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:20.214316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.215051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:17:20.215051 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.215051 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:20.215051 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.215051 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.215051 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:20.215051 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.215051 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:20.216416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.217137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:20.218115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.218610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.219120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:20.219120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.219120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:20.219120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:20.219120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:20.219315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:20.219647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.220327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:17:20.220327 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.220327 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:20.220327 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.220327 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.220327 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:20.220327 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.220327 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:20.221139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.221598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:20.222093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.222514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:20.223083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.223083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:20.279756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:20.280085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:20.280745 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:20.281160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:20.281393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.282076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:20.282076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.282076 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:20.282076 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.282076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.282076 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:20.282076 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.282076 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.282849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:20.283196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:25.077047 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:25.078579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:25.113873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:25.114068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.114750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:25.114750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.114750 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:25.114750 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.114750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.114750 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:25.114750 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.114750 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.115400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.116602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:25.208901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:25.210204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:25.210516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:25.210732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.211599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:25.211599 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.211599 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:25.211599 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.211599 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.211599 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:25.211599 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.211599 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.213362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.214445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:25.215502 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:25.215502 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:25.215502 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:25.215502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.215821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:25.215821 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.215821 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:25.215821 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.215821 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.215821 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:25.215821 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.215821 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:25.216886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:25.218020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:25.218020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.218020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:25.218020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.218020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.218020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:25.218020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.218020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:25.218678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.219268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.219833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:25.219833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.219833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:25.227744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:25.228336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:25.229087 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:25.229559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.230354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Traceback (most recent call last):
Mar 26 01:17:25.230354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.230354 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:25.230354 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.230354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.230354 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:25.230354 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.230354 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:25.231312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.231757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-12cd29e9-d90f-4be2-ad9e-7fe74ee8a38b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:25.232467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.233024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:25.233573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.233573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:25.280422 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:25.280968 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:25.281462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:25.281794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:25.282041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.282822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:25.282822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.282822 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:25.282822 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.282822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.282822 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:25.282822 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.282822 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.284780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:25.285686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:30.077446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:30.078670 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:30.080097 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:17:30.111160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:30.111812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.112652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:30.112652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.112652 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:30.112652 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.112652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.112652 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:30.112652 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.112652 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.113645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.114499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:30.209752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:30.210953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:30.211395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:30.211710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.212709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:30.212709 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.212709 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:30.212709 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.212709 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.212709 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:30.212709 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.212709 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.214274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.215012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:30.215864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:30.215864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:30.215864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:30.216265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.216771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:17:30.216771 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.216771 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:30.216771 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.216771 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.216771 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:30.216771 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.216771 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:30.218048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.218570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:30.219555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.220249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.220747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:30.220747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.220747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:30.228306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:30.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:30.229054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:30.229343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.229923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Traceback (most recent call last):
Mar 26 01:17:30.229923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.229923 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:30.229923 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.229923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.229923 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:30.229923 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.229923 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:30.230945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.231341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-c733f6c7-c753-4794-bf4e-134f9f9a3df0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:30.232072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.232510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:30.233272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.233272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:30.281126 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:30.281581 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:30.282303 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:30.282803 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:30.283150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.284083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:30.284083 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.284083 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:30.284083 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.284083 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.284083 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:30.284083 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.284083 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.285620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:30.286457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:35.078110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:35.079661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:35.113576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:35.114075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.115159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:35.115159 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.115159 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:35.115159 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.115159 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.115159 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:35.115159 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.115159 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.116290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.117334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:35.210444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:35.211723 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:35.211957 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:35.212272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.213182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:35.213182 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.213182 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:35.213182 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.213182 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.213182 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:35.213182 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.213182 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.214797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.215726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:35.216702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:35.216702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:35.216702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:35.216891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.218010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:17:35.218010 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.218010 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:35.218010 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.218010 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.218010 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:35.218010 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.218010 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:35.219582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.220441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:35.221341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.222199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.223094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:35.223094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.223094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:35.228757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:35.229464 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:35.229963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:35.230358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.231213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:17:35.231213 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.231213 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:35.231213 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.231213 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.231213 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:35.231213 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.231213 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:35.232529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.232935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:35.234462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.234927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:35.235401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.235401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:35.281643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:35.282270 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:35.282775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:35.283152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:35.283375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.284158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:35.284158 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.284158 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:35.284158 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.284158 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.284158 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:35.284158 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.284158 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.285775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:35.286535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:40.078842 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:40.080565 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:40.117436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:40.117650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.118351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:40.118351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.118351 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:40.118351 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.118351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.118351 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:40.118351 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.118351 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.119782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.120616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:40.211121 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:40.212194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:40.212535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:40.212915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.213452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:40.213452 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.213452 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:40.213452 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.213452 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.213452 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:40.213452 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.213452 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.215122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.215568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:40.216022 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:40.216088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:40.216467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:40.216639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.217225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:17:40.217225 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.217225 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:40.217225 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.217225 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.217225 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:40.217225 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.217225 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:40.218541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.218918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:40.220578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.221038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.221556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:40.221556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.221556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:40.229225 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:40.229713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:40.230159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:40.230404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.231015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:17:40.231015 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.231015 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:40.231015 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.231015 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.231015 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:40.231015 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.231015 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:40.232540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.233552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:40.234815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.235802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:40.236616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.236616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:40.282241 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:40.282773 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:40.283119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:40.283324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:40.283459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.284006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:40.284006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.284006 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:40.284006 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.284006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.284006 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:40.284006 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.284006 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.285643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:40.286481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:45.081913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:45.083613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:45.119008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:45.119437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.120288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:45.120288 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.120288 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:45.120288 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.120288 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.120288 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:45.120288 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.120288 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.121642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.122219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:45.211804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:45.212690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:45.212998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:45.213207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.213719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:45.213719 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.213719 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:45.213719 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.213719 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.213719 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:45.213719 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.213719 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.214921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.215824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:45.216702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:45.216702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:45.216702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:45.216702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.217248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:17:45.217248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.217248 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:45.217248 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.217248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.217248 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:45.217248 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.217248 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:45.218082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.218927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:45.219544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.220401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.221350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:45.221350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.221350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:45.229620 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:45.230204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:45.230786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:45.231264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.232545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:17:45.232545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.232545 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:45.232545 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.232545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.232545 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:45.232545 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.232545 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:45.233819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.234589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:45.235482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.236381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:45.237277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.237277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:45.282812 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:45.283408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:45.283897 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:45.284318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:45.284542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.285269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:45.285269 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.285269 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:45.285269 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.285269 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.285269 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:45.285269 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.285269 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.286893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:45.287814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:50.082560 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:50.083755 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:50.119407 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:50.119778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.120642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:50.120642 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.120642 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:50.120642 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.120642 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.120642 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:50.120642 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.120642 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.121513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.122399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:50.212318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:50.213663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:50.214518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:50.214761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.215552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:50.215552 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.215552 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:50.215552 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.215552 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.215552 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:50.215552 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.215552 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.217510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.218308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:50.219122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:50.219122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:50.219122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:50.219325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.220011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:17:50.220011 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.220011 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:50.220011 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.220011 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.220011 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:50.220011 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.220011 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:50.220924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.221625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:50.221983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.222450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.223041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:50.223041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.223041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:50.230088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:50.230654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:50.231259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:50.231779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.232847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Traceback (most recent call last):
Mar 26 01:17:50.232847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.232847 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:50.232847 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.232847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.232847 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:50.232847 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.232847 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:50.234586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.235293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-e39599db-81b4-46ea-a8bf-fc71f7618e73 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:50.237151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.237891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:50.238722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.238722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:50.283308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:50.283838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:50.284236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:50.284507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:50.284665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:50.285198 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.285198 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:50.285198 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.285198 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.285198 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:50.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.286226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:50.287290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:55.083265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:55.084826 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:55.117105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:55.117540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.118474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:17:55.118474 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.118474 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:55.118474 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.118474 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.118474 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:55.118474 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.118474 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.119505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.120493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:55.213081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:55.214452 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:55.214779 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:55.215006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.215907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:55.215907 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.215907 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:55.215907 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.215907 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.215907 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:55.215907 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.215907 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.217655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.218539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:55.219380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:55.219380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:17:55.219380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:55.219595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.220343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:17:55.220343 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.220343 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:55.220343 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.220343 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.220343 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:55.220343 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.220343 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:55.221558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.222453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:17:55.222962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.223697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.224549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:55.224549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.224549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:55.230642 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:55.231198 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:55.231849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:55.232384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.233239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:17:55.233239 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.233239 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:55.233239 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.233239 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.233239 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:55.233239 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.233239 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:17:55.234499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.234960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:17:55.236160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.236620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:55.237169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.237169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:17:55.284298 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:17:55.285248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:17:55.286022 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:55.286608 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:17:55.286952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.287824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:17:55.287824 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.287824 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:17:55.287824 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.287824 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.287824 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:17:55.287824 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.287824 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.288909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:17:55.289698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:00.084182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:00.085727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:00.116727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:00.117126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.117921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:18:00.117921 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.117921 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:00.117921 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.117921 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.117921 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:00.117921 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.117921 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.118820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.119269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:00.213721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:00.214899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:00.215194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:00.215470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.216213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:00.216213 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.216213 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:00.216213 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.216213 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.216213 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:00.216213 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.216213 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.217684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.218623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:00.219714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:00.219714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:00.219947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:00.220182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.221100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:00.221100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.221100 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:00.221100 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.221100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.221100 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:00.221100 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.221100 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:00.222527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.223398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:00.224139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.225005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.225910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:00.225910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.225910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:00.231082 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:00.231398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:00.231730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:00.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.232867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:18:00.232867 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.232867 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:00.232867 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.232867 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.232867 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:00.232867 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.232867 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:00.234191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.234582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:00.235857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.236691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:00.237499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.237499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:00.284788 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:00.285329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:00.285813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:00.286216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:00.286443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.287153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:00.287153 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.287153 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:00.287153 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.287153 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.287153 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:00.287153 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.287153 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.288923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:00.289748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:05.084655 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:05.086194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:05.123056 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:05.123406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.124287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:18:05.124287 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.124287 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:05.124287 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.124287 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.124287 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:05.124287 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.124287 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.125504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.126398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:05.214358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:05.215318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:05.215510 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:05.215640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.216717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:05.216717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.216717 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:05.216717 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.216717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.216717 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:05.216717 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.216717 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.219022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.220115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:05.220909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:05.220909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:05.220909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:05.220909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.221623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:05.221623 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.221623 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:05.221623 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.221623 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.221623 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:05.221623 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.221623 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:05.223219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.224154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:05.225442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.225930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.226548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:05.226548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.226548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:05.231425 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:05.231898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:05.232457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:05.232861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.233632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:18:05.233632 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.233632 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:05.233632 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.233632 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.233632 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:05.233632 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.233632 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:05.234986 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.235437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:05.236770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.237305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:05.237806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.237806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:05.285260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:05.285555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:05.286311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:05.286854 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:05.286961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.287654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:05.287654 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.287654 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:05.287654 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.287654 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.287654 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:05.287654 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.287654 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.288628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:05.289507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:10.085244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:10.086611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:10.125102 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:10.125501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.126500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Traceback (most recent call last):
Mar 26 01:18:10.126500 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.126500 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:10.126500 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.126500 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.126500 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:10.126500 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.126500 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fbe5df90-e829-4ac0-ba2f-8e20b6fb30d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.127599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.128229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:10.215059 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:10.216145 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:10.216512 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:10.216768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.217602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:10.217602 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.217602 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:10.217602 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.217602 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.217602 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:10.217602 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.217602 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.219324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.220227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:10.221207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:10.221207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:10.221808 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:10.222110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.223075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:10.223075 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.223075 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:10.223075 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.223075 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.223075 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:10.223075 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.223075 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:10.224690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.225590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:10.226434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.226940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.227463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:10.227463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.227463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:10.231899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:10.232309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:10.232847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:10.233311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.234082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:18:10.234082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.234082 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:10.234082 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.234082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.234082 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:10.234082 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.234082 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:10.234913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.235373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:10.235936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.236459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:10.236923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.236923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:10.285702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:10.286050 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:10.286486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:10.286747 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:10.286895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.287428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:10.287428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.287428 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:10.287428 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.287428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.287428 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:10.287428 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.287428 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.288953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:10.289562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:15.085910 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:15.087514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fdbb945e-eff7-40db-88ba-880fa2be2fa6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:15.118754 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:15.119009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.119923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fdbb945e-eff7-40db-88ba-880fa2be2fa6 None None] Traceback (most recent call last):
Mar 26 01:18:15.119923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.119923 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:15.119923 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.119923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.119923 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:15.119923 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.119923 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fdbb945e-eff7-40db-88ba-880fa2be2fa6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.120828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.121726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:15.215800 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:15.216781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:15.216988 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:15.217160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.217731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:15.217731 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.217731 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:15.217731 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.217731 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.217731 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:15.217731 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.217731 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.219168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.220161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:15.221053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:15.221559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:15.221988 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:15.222242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.222741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:15.222741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.222741 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:15.222741 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.222741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.222741 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:15.222741 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.222741 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:15.223678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.224307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:15.224691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.225233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.226036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:15.226036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.226036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:15.232385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:15.232882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:15.233341 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:15.233631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.234443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:18:15.234443 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.234443 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:15.234443 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.234443 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.234443 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:15.234443 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.234443 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:15.235918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.236349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:15.237959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.238478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:15.239008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.239008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:15.286377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:15.287003 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:15.287831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:15.288412 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:15.288846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.289747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:15.289747 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.289747 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:15.289747 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.289747 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.289747 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:15.289747 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.289747 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.290930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:15.291467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:20.086595 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:20.088412 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fdbb945e-eff7-40db-88ba-880fa2be2fa6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:20.123209 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:20.123462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.124242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fdbb945e-eff7-40db-88ba-880fa2be2fa6 None None] Traceback (most recent call last):
Mar 26 01:18:20.124242 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.124242 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:20.124242 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.124242 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.124242 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:20.124242 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.124242 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-fdbb945e-eff7-40db-88ba-880fa2be2fa6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.125545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.126450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:20.216528 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:20.217939 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:20.218307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:20.218584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.219530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:20.219530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.219530 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:20.219530 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.219530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.219530 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:20.219530 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.219530 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.221455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.222589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:20.223418 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:20.223418 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:20.223418 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:20.223418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.224151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:20.224151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.224151 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:20.224151 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.224151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.224151 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:20.224151 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.224151 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:20.225397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:20.226029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:20.226029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.226029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:20.226029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.226029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.226029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:20.226029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.226029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:20.226593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.227110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.227590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:20.227590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.227590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:20.232797 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:20.233307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:20.233958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:20.234387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.234873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Traceback (most recent call last):
Mar 26 01:18:20.234873 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.234873 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:20.234873 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.234873 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.234873 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:20.234873 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.234873 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:20.236167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.236635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-841874aa-14fa-41a4-8e4e-17a88c5971a6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:20.237749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.238238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:20.238697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.238697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:20.287017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:20.287510 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:20.288053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:20.288413 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:20.288635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.289339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:20.289339 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.289339 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:20.289339 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.289339 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.289339 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:20.289339 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.289339 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.290547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:20.291077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:24.806051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:24.815875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 26 01:18:24.816056 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105681) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}}
Mar 26 01:18:24.816188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:18:24.847246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, resource={'id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'name': 'private', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['02d96a4b-4c93-4b95-ad0f-f11d085f0699', '0503b55a-1b69-49b5-b418-16ba37be90e9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T00:58:10Z', 'updated_at': '2026-03-26T00:58:13Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4449}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.847613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.847990 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6fe31530-ff9f-476b-aea1-4804059b76cd, external_ids_update=None, columns={'tag': 4449}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.848152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.849124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:18:24.849124 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:24.849124 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:24.849124 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:24.849124 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:24.849124 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:24.849124 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.849124 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:24.850720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:18:24.851687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:24.852535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:18:24.852535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.852535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:18:24.852535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:18:24.881597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-db438906-c74f-4dc8-883c-6e56962e551b, resource={'id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'name': 'public', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['1c3e4583-6e63-4953-8050-3341662a5ec6', '23682989-62bf-4e07-a74f-0cb01077d5ac'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-26T00:58:16Z', 'updated_at': '2026-03-26T00:58:25Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.881926 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-db438906-c74f-4dc8-883c-6e56962e551b, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.882347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-200618ec-0540-41b0-9429-dc7dfc74abb0, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.882582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.883313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:18:24.883313 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:24.883313 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:24.883313 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:24.883313 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:24.883313 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:24.883313 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.883313 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:24.884245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:18:24.884749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:24.885435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:18:24.885435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.885435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:18:24.885435 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:18:24.910777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, resource={'id': 'f7e819e4-7101-412b-b047-baa5e16b2a52', 'name': 'admin_net', 'tenant_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'project_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f2e900ee-4d88-4151-8e50-661745950632'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:14Z', 'updated_at': '2026-03-26T01:02:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4803}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.911014 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.911365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d43d91f1-f24a-411f-bae7-af85048f95b9, external_ids_update=None, columns={'tag': 4803}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.911476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.912241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:18:24.912241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:24.912241 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:24.912241 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:24.912241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:24.912241 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:24.912241 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.912241 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:24.913497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:18:24.913772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:24.914088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:18:24.914088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.914088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:18:24.914088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:18:24.942797 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, resource={'id': 'f481c2f9-5052-46d5-821a-6c8712ec70ec', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['329ea1d3-f656-45b1-bf23-1d927d4578ea'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:50Z', 'updated_at': '2026-03-26T01:02:51Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19903}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.943159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.943529 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3aa5be7a-5c67-4f1a-bdf2-7bf6912e871d, external_ids_update=None, columns={'tag': 19903}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:24.943636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.944468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:18:24.944468 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:24.944468 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:24.944468 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:24.944468 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:24.944468 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:24.944468 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.944468 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:24.945605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:18:24.946208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:24.946771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:18:24.946771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:24.946771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:18:24.946771 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:18:25.017207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, resource={'id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'enable_snat': True}, 'gw_port_id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-ryt6n', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'ip_address': '172.24.5.142', 'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac'}, {'ip_address': '2001:db8::16a', 'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 3, 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8'}, resource_type=routers, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.017486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.017815 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.018090 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.018090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.018938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:18:25.018938 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.018938 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:25.018938 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.018938 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.018938 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:25.018938 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.018938 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:25.019317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Unable to update router e4efd798-3c25-4ea9-bd45-fa7a05aeaef9. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 26 01:18:25.020803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:18:25.021382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:18:25.022169 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:18:25.089938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:25.095360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:25.157800 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.158991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.161857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:18:25.161857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.161857 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:25.161857 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.161857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.161857 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:25.161857 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.161857 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.165016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.165916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:25.168077 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, resource={'id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'name': '', 'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'project_id': '', 'mac_address': 'fa:16:3e:4e:d4:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160849', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.168336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::16a/64', '172.24.5.142/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.168651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=14a0aa42-40aa-4508-9af3-cffdffef9818, lrouter_port=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.168886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.169655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:18:25.169655 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.169655 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:25.169655 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.169655 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.169655 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:25.169655 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.169655 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:25.170921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.171503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.172113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:18:25.172113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.172113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:18:25.172113 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Synchronization completed (took 0.35 seconds)
Mar 26 01:18:25.217408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:25.218866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.219105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.219264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.220034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:25.220034 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.220034 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:25.220034 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.220034 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.220034 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:25.220034 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.220034 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.221921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.223087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:25.224094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:25.224203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:25.224929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.224929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.225672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:18:25.225672 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.225672 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:25.225672 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.225672 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.225672 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:25.225672 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.225672 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:25.227180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.227918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:25.229027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.229782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.230645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:25.230645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.230645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:25.233357 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:25.233719 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:25.234084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.234310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.234770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:25.234770 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.234770 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:25.234770 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.234770 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.234770 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:25.234770 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.234770 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:25.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.235957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:25.235957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.235957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:25.235957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.235957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.235957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:25.235957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.235957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:25.236727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.237069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:25.237399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.237399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:25.288012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:25.288405 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:25.288647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.288859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:25.289003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.289463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:25.289463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.289463 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:25.289463 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.289463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.289463 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:25.289463 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.289463 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.290391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:25.291201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:30.091857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:30.093447 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:30.095070 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=434 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:18:30.131084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:30.131361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.131892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:18:30.131892 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.131892 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:30.131892 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.131892 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.131892 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:30.131892 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.131892 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.132996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.133785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:30.218296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:30.219600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:30.219944 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:30.220283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.221256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:30.221256 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.221256 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:30.221256 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.221256 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.221256 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:30.221256 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.221256 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.222924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.223825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:30.224769 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:30.224769 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:30.224769 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:30.224769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.225466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:18:30.225466 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.225466 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:30.225466 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.225466 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.225466 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:30.225466 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.225466 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:30.226909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.227706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:30.228190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.229213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.230066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:30.230066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.230066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:30.233810 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:30.234243 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:30.234657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:30.235027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.235665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:30.235665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.235665 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:30.235665 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.235665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.235665 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:30.235665 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.235665 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:30.236879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:30.237335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:30.237335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.237335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:30.237335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.237335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.237335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:30.237335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.237335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:30.238534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.239358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:30.240248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.240248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:30.288746 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:30.289248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:30.289631 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:30.289947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:30.290221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.290906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:30.290906 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.290906 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:30.290906 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.290906 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.290906 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:30.290906 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.290906 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.292461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:30.293664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:35.092815 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:35.094566 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:35.132817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:35.133361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.134338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:18:35.134338 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.134338 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:35.134338 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.134338 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.134338 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:35.134338 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.134338 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.135288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.136238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:35.219490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:35.221163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:35.222784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:35.222784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.224169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:35.224169 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.224169 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:35.224169 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.224169 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.224169 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:35.224169 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.224169 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.225764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.226923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:35.227855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:35.227855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:35.227855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:35.227855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.228235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:18:35.228235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.228235 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:35.228235 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.228235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.228235 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:35.228235 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.228235 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:35.229373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.229848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:35.230300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.230652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.231020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:35.231020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.231020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:35.234138 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:35.234370 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:35.234644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:35.234870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.235303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:35.235303 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.235303 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:35.235303 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.235303 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.235303 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:35.235303 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.235303 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:35.236050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:35.236411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:35.236411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.236411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:35.236411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.236411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.236411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:35.236411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.236411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:35.237134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.237460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:35.237837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.237837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:35.289165 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:35.289713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:35.290221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:35.290598 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:35.290831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.291640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:35.291640 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.291640 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:35.291640 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.291640 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.291640 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:35.291640 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.291640 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.293282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:35.293810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:40.093714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:40.094910 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:40.122706 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:40.123122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.123827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:18:40.123827 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.123827 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:40.123827 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.123827 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.123827 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:40.123827 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.123827 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.124655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.125796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:40.220012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:40.221234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:40.221600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:40.221864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.222739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:40.222739 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.222739 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:40.222739 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.222739 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.222739 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:40.222739 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.222739 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.224491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.225282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:40.226842 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:40.227484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:40.228155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:40.228546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.229636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:18:40.229636 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.229636 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:40.229636 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.229636 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.229636 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:40.229636 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.229636 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:40.230734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.231232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:40.232078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.233062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:40.233062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.233062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:40.234583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:40.235049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:40.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:40.235917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.236637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:40.236637 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.236637 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:40.236637 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.236637 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.236637 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:40.236637 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.236637 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:40.237889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:40.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:40.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:40.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:40.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:40.239553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.240028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:40.240536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.240536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:40.290049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:40.290827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:40.291472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:40.291879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:40.292150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.293024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:40.293024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.293024 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:40.293024 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.293024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.293024 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:40.293024 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.293024 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.294669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:40.295260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:45.094449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:45.096106 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:45.134727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:45.135171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.136103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:18:45.136103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.136103 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:45.136103 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.136103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.136103 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:45.136103 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.136103 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.137153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.138243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:45.220683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:45.221677 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:45.221914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:45.222129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.222881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:45.222881 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.222881 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:45.222881 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.222881 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.222881 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:45.222881 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.222881 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.224711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.225774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:45.227334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:45.227935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:45.228586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:45.228867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.229875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:18:45.229875 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.229875 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:45.229875 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.229875 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.229875 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:45.229875 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.229875 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:45.231215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.232535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:45.232920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.233566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.234249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:45.234249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.234249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:45.234922 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:45.235146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:45.235401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:45.235653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.236061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:45.236061 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.236061 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:45.236061 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.236061 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.236061 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:45.236061 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.236061 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:45.236732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:45.237444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:45.237444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.237444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:45.237444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.237444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.237444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:45.237444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.237444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:45.237796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.238519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:45.239311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.239311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:45.290663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:45.291045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:45.291415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:45.291842 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:45.292220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.292904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:45.292904 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.292904 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:45.292904 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.292904 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.292904 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:45.292904 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.292904 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.294466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:45.295251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:50.095199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:50.096586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:50.132443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:50.132820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.133680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:18:50.133680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.133680 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:50.133680 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.133680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.133680 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:50.133680 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.133680 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.135000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.136101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:50.221375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:50.222633 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:50.223103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:50.223103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.224316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:50.224316 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.224316 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:50.224316 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.224316 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.224316 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:50.224316 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.224316 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.226286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.227109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:50.227789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:50.228218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:50.228591 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:50.228875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.229571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:18:50.229571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.229571 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:50.229571 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.229571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.229571 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:50.229571 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.229571 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:50.230853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.231331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:50.232127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.232908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.233426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:50.233426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.233426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:50.235325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:50.235545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:50.235777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:50.235989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.236390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:50.236390 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.236390 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:50.236390 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.236390 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.236390 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:50.236390 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.236390 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:50.237041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:50.237481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:50.237481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.237481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:50.237481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.237481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.237481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:50.237481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.237481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:50.237940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.238397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:50.238863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.238863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:50.291225 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:50.291859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:50.292515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:50.292914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:50.293255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.294156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:50.294156 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.294156 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:50.294156 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.294156 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.294156 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:50.294156 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.294156 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.295835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:50.296594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:55.096103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:55.097747 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:55.136135 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:55.136534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.137479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:18:55.137479 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.137479 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:55.137479 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.137479 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.137479 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:55.137479 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.137479 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.138787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.139400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:55.221903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:55.223135 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:55.223494 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:55.223728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.224580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:55.224580 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.224580 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:55.224580 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.224580 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.224580 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:55.224580 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.224580 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.226481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.227040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:55.228147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:55.228608 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:18:55.229050 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:55.229355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.230140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:18:55.230140 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.230140 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:55.230140 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.230140 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.230140 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:55.230140 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.230140 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:55.231310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.231793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:18:55.232811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.233293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.233773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:55.233773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.233773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:55.235625 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:55.235938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:55.236332 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:55.236654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.237284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:18:55.237284 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.237284 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:55.237284 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.237284 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.237284 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:55.237284 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.237284 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:18:55.238322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:55.238745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:18:55.238745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.238745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:18:55.238745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.238745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.238745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:18:55.238745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.238745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:18:55.239508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.240068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:55.240537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.240537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:18:55.291661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:18:55.292248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:18:55.292666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:55.292990 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:18:55.293232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.293909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:18:55.293909 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.293909 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:18:55.293909 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.293909 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.293909 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:18:55.293909 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.293909 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.295682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:18:55.296141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:00.096656 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:00.097521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:00.123264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:00.123538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.124171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:00.124171 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.124171 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:00.124171 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.124171 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.124171 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:00.124171 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.124171 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.125012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.125838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:00.222811 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:00.223900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:00.224207 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:00.224453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.225210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:00.225210 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.225210 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:00.225210 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.225210 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.225210 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:00.225210 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.225210 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.226997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.227713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:00.228527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:00.228923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:00.229345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:00.229612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.230323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:00.230323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.230323 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:00.230323 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.230323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.230323 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:00.230323 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.230323 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:00.231501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.232253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:00.233007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.233499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.234064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:00.234064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.234064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:00.236005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:00.236290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:00.236687 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:00.237068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.237633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:00.237633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.237633 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:00.237633 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.237633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.237633 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:00.237633 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.237633 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:00.238748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:00.239306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:00.239306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.239306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:00.239306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.239306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.239306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:00.239306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.239306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:00.241147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.241637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:00.242208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.242208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:00.292239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:00.292682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:00.293166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:00.293520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:00.293745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.294488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:00.294488 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.294488 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:00.294488 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.294488 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.294488 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:00.294488 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.294488 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.296003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:00.296890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:05.097314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:05.098095 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:05.123641 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:05.124016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.124837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:05.124837 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.124837 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:05.124837 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.124837 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.124837 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:05.124837 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.124837 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.125748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.126735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:05.223322 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:05.224500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:05.224794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:05.225063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.225990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:05.225990 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.225990 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:05.225990 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.225990 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.225990 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:05.225990 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.225990 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.227797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.228606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:05.229427 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:05.229680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:05.230116 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:05.230401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.231158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:05.231158 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.231158 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:05.231158 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.231158 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.231158 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:05.231158 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.231158 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:05.232844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.233667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:05.234566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.235309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.235791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:05.235791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.235791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:05.236466 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:05.236876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:05.237137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:05.237351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.237965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:05.237965 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.237965 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:05.237965 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.237965 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.237965 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:05.237965 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.237965 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:05.238685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:05.239538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:05.239538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.239538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:05.239538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.239538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.239538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:05.239538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.239538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:05.239748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.240299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:05.241196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.241196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:05.292748 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:05.293278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:05.293915 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:05.294407 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:05.295025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.296126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:05.296126 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.296126 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:05.296126 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.296126 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.296126 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:05.296126 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.296126 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.297456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:05.298203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:10.098057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:10.099583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:10.137300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:10.137786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.138645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:10.138645 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.138645 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:10.138645 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.138645 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.138645 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:10.138645 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.138645 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.139540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.140316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:10.223832 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:10.225196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:10.225630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:10.225910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.226795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:10.226795 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.226795 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:10.226795 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.226795 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.226795 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:10.226795 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.226795 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.228609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.229372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:10.230279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:10.230347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:10.230772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:10.231080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.231529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:10.231529 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.231529 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:10.231529 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.231529 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.231529 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:10.231529 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.231529 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:10.232876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.233695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:10.234468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.235299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.236249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:10.236249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.236249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:10.236940 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:10.237450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:10.237858 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:10.238096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.238602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:10.238602 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.238602 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:10.238602 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.238602 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.238602 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:10.238602 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.238602 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:10.239826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:10.240297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:10.240297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.240297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:10.240297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.240297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.240297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:10.240297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.240297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:10.241433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.241959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:10.242471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.242471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:10.293454 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:10.294266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:10.294789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:10.295481 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:10.295481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.296350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:10.296350 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.296350 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:10.296350 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.296350 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.296350 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:10.296350 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.296350 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.298111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:10.299157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:15.098712 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:15.100561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:15.138528 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:15.138783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.139656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:15.139656 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.139656 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:15.139656 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.139656 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.139656 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:15.139656 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.139656 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.140576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.141182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:15.224461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:15.225509 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:15.225846 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:15.226131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.226898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:15.226898 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.226898 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:15.226898 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.226898 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.226898 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:15.226898 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.226898 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.228039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.229052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:15.230332 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:15.230758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:15.231236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:15.231522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.232255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:15.232255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.232255 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:15.232255 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.232255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.232255 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:15.232255 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.232255 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:15.233458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.234622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:15.235117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.235889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.236775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:15.236775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.236775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:15.237424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:15.237797 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:15.238282 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:15.238621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.239288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:15.239288 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.239288 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:15.239288 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.239288 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.239288 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:15.239288 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.239288 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:15.240566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:15.241084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:15.241084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.241084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:15.241084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.241084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.241084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:15.241084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.241084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:15.242228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:15.243210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.243210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:15.293958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:15.294408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:15.294900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:15.295255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:15.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.296176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:15.296176 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.296176 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:15.296176 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.296176 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.296176 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:15.296176 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.296176 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.297672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:15.298499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:20.099408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:20.101023 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:20.133935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:20.134351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.135185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:20.135185 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.135185 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:20.135185 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.135185 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.135185 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:20.135185 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.135185 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.136301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.137486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:20.225119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:20.226235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:20.226508 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:20.226732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.227566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:20.227566 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.227566 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:20.227566 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.227566 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.227566 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:20.227566 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.227566 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.229228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.230110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:20.230931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:20.231311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:20.231727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:20.231890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.232537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:20.232537 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.232537 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:20.232537 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.232537 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.232537 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:20.232537 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.232537 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:20.234372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.235554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:20.236632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.237464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.238157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:20.238157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.238157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:20.238157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:20.238385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:20.239058 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:20.239432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.240196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:20.240196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.240196 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:20.240196 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.240196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.240196 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:20.240196 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.240196 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:20.241571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:20.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:20.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:20.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:20.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:20.243556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.244056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:20.244516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.244516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:20.294405 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:20.294405 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:20.295277 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:20.295632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:20.295801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.298615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:20.299468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:25.100066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:25.101640 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:25.130518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:25.130915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.131836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:25.131836 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.131836 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:25.131836 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.131836 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.131836 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:25.131836 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.131836 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.132815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.133732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:25.225677 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:25.227373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:25.227704 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:25.227968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.228866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:25.228866 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.228866 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:25.228866 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.228866 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.228866 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:25.228866 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.228866 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.230473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.231262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:25.232029 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:25.232029 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:25.232595 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:25.232868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.233671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:25.233671 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.233671 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:25.233671 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.233671 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.233671 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:25.233671 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.233671 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:25.234993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.235475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:25.236500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.237032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.237699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:25.237699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.237699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:25.238199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:25.238523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:25.238958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:25.239312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.240035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:25.240035 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.240035 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:25.240035 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.240035 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.240035 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:25.240035 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.240035 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:25.240936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:25.241672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:25.241672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.241672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:25.241672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.241672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.241672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:25.241672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.241672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:25.242379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.242931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:25.243490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.243490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:25.294911 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:25.295655 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:25.296714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:25.297482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:25.297886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.298836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:25.298836 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.298836 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:25.298836 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.298836 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.298836 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:25.298836 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.298836 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.300549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:25.301445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:30.100632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:30.102128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:30.103646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:19:30.139731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:30.139990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.140716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:30.140716 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.140716 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:30.140716 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.140716 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.140716 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:30.140716 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.140716 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.141847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.143234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:30.226649 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:30.227941 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:30.228363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:30.228708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.229815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:30.229815 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.229815 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:30.229815 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.229815 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.229815 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:30.229815 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.229815 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.231625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.232509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:30.233361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:30.233361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:30.233361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:30.233361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.234078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:30.234078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.234078 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:30.234078 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.234078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.234078 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:30.234078 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.234078 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:30.235052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.235811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:30.236272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.237000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.237801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:30.237801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.237801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:30.238646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:30.239108 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:30.239492 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:30.239765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.240303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:30.240303 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.240303 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:30.240303 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.240303 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.240303 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:30.240303 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.240303 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:30.241463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:30.242150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:30.242150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.242150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:30.242150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.242150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.242150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:30.242150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.242150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:30.242958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.243815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:30.244408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.244408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:30.295460 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:30.295994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:30.296727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:30.297171 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:30.297370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.298244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:30.298244 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.298244 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:30.298244 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.298244 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.298244 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:30.298244 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.298244 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.300166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:30.300930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:35.101374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:35.102926 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:35.134295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:35.134701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.135576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:35.135576 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.135576 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:35.135576 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.135576 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.135576 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:35.135576 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.135576 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.136428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.137288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:35.227283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:35.228603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:35.228920 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:35.229186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.230008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:35.230008 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.230008 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:35.230008 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.230008 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.230008 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:35.230008 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.230008 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.231713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.232514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:35.233350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:35.233350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:35.233350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:35.233616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.234334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:35.234334 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.234334 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:35.234334 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.234334 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.234334 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:35.234334 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.234334 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:35.235512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.236277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:35.236972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.237647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.238161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:35.238161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.238161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:35.239067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:35.239561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:35.240092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:35.240430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.241114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:35.241114 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.241114 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:35.241114 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.241114 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.241114 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:35.241114 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.241114 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:35.242547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:35.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:35.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:35.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:35.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:35.244466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.244950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:35.245489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.245489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:35.296112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:35.296736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:35.297163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:35.297843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:35.297843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.298407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:35.298407 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.298407 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:35.298407 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.298407 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.298407 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:35.298407 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.298407 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.299733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:35.300294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:40.102011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:40.103186 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:40.136867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:40.137256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.138086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:40.138086 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.138086 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:40.138086 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.138086 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.138086 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:40.138086 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.138086 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.139341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.140411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:40.227854 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:40.229230 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:40.229552 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:40.229778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.230812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:40.230812 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.230812 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:40.230812 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.230812 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.230812 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:40.230812 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.230812 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.232289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.233037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:40.233473 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:40.233473 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:40.233558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:40.233794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.234540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:40.234540 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.234540 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:40.234540 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.234540 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.234540 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:40.234540 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.234540 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:40.235814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.236681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:40.237353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.238403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.239070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:40.239070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.239070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:40.239667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:40.240117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:40.240550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:40.240872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.241565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:40.241565 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.241565 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:40.241565 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.241565 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.241565 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:40.241565 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.241565 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:40.243001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:40.243540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:40.243540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.243540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:40.243540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.243540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.243540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:40.243540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.243540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:40.244665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.245248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:40.245750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.245750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:40.296616 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:40.297678 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:40.298322 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:40.298680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:40.298883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.299712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:40.299712 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.299712 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:40.299712 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.299712 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.299712 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:40.299712 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.299712 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.301370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:40.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:45.102683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:45.104144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:45.140823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:45.141258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.142202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:45.142202 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.142202 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:45.142202 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.142202 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.142202 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:45.142202 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.142202 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.143263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.144221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:45.228532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:45.229682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:45.230320 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:45.230320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.231148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:45.231148 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.231148 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:45.231148 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.231148 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.231148 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:45.231148 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.231148 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.232337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.233105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:45.233860 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:45.233860 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:45.234089 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:45.234269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.234838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:45.234838 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.234838 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:45.234838 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.234838 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.234838 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:45.234838 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.234838 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:45.236133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.236816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:45.237634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.238362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.239195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:45.239195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.239195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:45.240255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:45.240813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:45.241334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:45.241679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.242389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:45.242389 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.242389 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:45.242389 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.242389 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.242389 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:45.242389 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.242389 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:45.243643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:45.244107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:45.244107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.244107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:45.244107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.244107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.244107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:45.244107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.244107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:45.245313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.245739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:45.246266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.246266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:45.297501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:45.297993 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:45.298314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:45.298588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:45.298730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.299275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:45.299275 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.299275 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:45.299275 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.299275 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.299275 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:45.299275 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.299275 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.300293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:45.301045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:50.103784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:50.105841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:50.143513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:50.143779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.144553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:50.144553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.144553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:50.144553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.144553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.144553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:50.144553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.144553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.145830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.146909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:50.229240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:50.230446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:50.230650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:50.230813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.231557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:50.231557 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.231557 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:50.231557 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.231557 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.231557 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:50.231557 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.231557 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.233216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.234050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:50.235002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:50.235002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:50.235002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:50.235002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.235411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:50.235411 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.235411 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:50.235411 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.235411 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.235411 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:50.235411 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.235411 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:50.236852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.237794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:50.238640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.239764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.240314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:50.240314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.240314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:50.240813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:50.241388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:50.241794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:50.242050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.242599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:50.242599 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.242599 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:50.242599 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.242599 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.242599 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:50.242599 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.242599 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:50.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:50.244352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:50.244352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.244352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:50.244352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.244352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.244352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:50.244352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.244352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:50.245580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.246620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:50.247216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.247216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:50.298200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:50.298762 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:50.299202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:50.299459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:50.299618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.300291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:50.300291 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.300291 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:50.300291 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.300291 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.300291 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:50.300291 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.300291 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.301842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:50.302802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:55.104917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:55.106420 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:55.140743 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:55.141020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.141904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:19:55.141904 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.141904 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:55.141904 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.141904 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.141904 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:55.141904 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.141904 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.142791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.143857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:55.229868 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:55.231161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:55.231470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:55.231717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.232628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:55.232628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.232628 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:55.232628 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.232628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.232628 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:55.232628 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.232628 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.234409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.235310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:55.236119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:55.236119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:19:55.236119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:55.236119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.236865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:19:55.236865 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.236865 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:55.236865 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.236865 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.236865 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:55.236865 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.236865 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:55.238236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.238720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:19:55.239730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.240264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.240772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:55.240772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.240772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:55.241266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:55.241646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:55.242041 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:55.242243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.242711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:19:55.242711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.242711 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:55.242711 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.242711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.242711 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:55.242711 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.242711 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:19:55.243927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:55.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:19:55.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:19:55.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:19:55.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.244500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:19:55.245732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.246225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:55.247197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.247197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:19:55.298730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:19:55.299403 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:19:55.299885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:55.300250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:19:55.300459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.301316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:19:55.301316 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.301316 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:19:55.301316 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.301316 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.301316 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:19:55.301316 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.301316 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.303084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:19:55.303902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:00.105581 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:00.106783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:00.141825 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:00.142232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.143103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:00.143103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.143103 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:00.143103 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.143103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.143103 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:00.143103 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.143103 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.143948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.145125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:00.230667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:00.232120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:00.232411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:00.232673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.233582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:00.233582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.233582 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:00.233582 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.233582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.233582 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:00.233582 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.233582 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.235369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.236265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.237377 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:00.238376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:00.239942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.241533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:00.242371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:00.243429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:00.243429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.244132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:00.244132 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.244132 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:00.244132 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.244132 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.244132 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:00.244132 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.244132 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:00.245202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:00.245823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:00.245823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.245823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:00.245823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.245823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.245823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:00.245823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.245823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:00.246374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.246828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:00.247210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.247210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:00.299609 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:00.300150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:00.300724 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:00.300997 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:00.301174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.301844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:00.301844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.301844 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:00.301844 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.301844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.301844 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:00.301844 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.301844 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.303692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:00.304489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:05.106256 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:05.107807 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:05.141881 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:05.141881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.143746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:05.143746 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.143746 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:05.143746 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.143746 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.143746 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:05.143746 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.143746 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.145075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.146022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:05.231495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:05.232631 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:05.233009 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:05.233263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.234121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:05.234121 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.234121 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:05.234121 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.234121 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.234121 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:05.234121 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.234121 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.235774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.236520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:05.237257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:05.237257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:05.237257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:05.237440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.238287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:05.238287 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.238287 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:05.238287 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.238287 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.238287 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:05.238287 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.238287 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:05.240185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.240891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:05.241904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.242614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.243393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:05.243393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.243393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:05.243393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:05.243393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:05.243393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:05.243715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.244597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:05.244597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.244597 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:05.244597 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.244597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.244597 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:05.244597 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.244597 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:05.245396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:05.246220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:05.246220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.246220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:05.246220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.246220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.246220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:05.246220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.246220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:05.246797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.247323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:05.247900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.247900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:05.300103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:05.300741 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:05.330833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:05.330833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:05.330833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.331811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:05.331811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.331811 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:05.331811 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.331811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.331811 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:05.331811 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.331811 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.333670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:05.334769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:10.106864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:10.107714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:10.138003 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:10.138287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.138990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:10.138990 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.138990 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:10.138990 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.138990 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.138990 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:10.138990 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.138990 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.140176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.141213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:10.232061 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:10.232924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:10.233111 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:10.233224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.233706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:10.233706 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.233706 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:10.233706 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.233706 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.233706 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:10.233706 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.233706 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.234886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.235719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:10.236527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:10.236719 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:10.236967 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:10.237113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.237509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:10.237509 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.237509 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:10.237509 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.237509 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.237509 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:10.237509 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.237509 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:10.238306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.238806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:10.239291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.239784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.240350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:10.240350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.240350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:10.242696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:10.242988 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:10.243264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:10.243416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:10.243875 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.243875 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:10.243875 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.243875 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.243875 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:10.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.243875 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:10.245361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:10.245870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:10.245870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.245870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:10.245870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.245870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.245870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:10.245870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.245870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:10.247046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.247546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:10.248115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.248115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:10.300830 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:10.301285 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:10.301572 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:10.301726 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:10.301816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.302253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:10.302253 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.302253 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:10.302253 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.302253 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.302253 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:10.302253 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.302253 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.303497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:10.304279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:15.107429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:15.108832 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:15.144198 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:15.144344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.144931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:15.144931 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.144931 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:15.144931 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.144931 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.144931 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:15.144931 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.144931 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.146055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.146607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:15.232607 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:15.233621 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:15.233886 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:15.234137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.234872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:15.234872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.234872 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:15.234872 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.234872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.234872 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:15.234872 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.234872 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.236491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.237267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:15.238026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:15.238026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:15.238026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:15.238026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.238489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:15.238489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.238489 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:15.238489 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.238489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.238489 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:15.238489 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.238489 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:15.239639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.240411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:15.241239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.241950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.242749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:15.242749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.242749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:15.243024 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:15.243167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:15.243490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:15.243713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.244238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:15.244238 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.244238 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:15.244238 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.244238 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.244238 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:15.244238 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.244238 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:15.244946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:15.245731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:15.245731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.245731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:15.245731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.245731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.245731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:15.245731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.245731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:15.246149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.246910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:15.247800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.247800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:15.301525 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:15.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:15.302539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:15.302891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:15.303143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.304027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:15.304027 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.304027 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:15.304027 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.304027 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.304027 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:15.304027 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.304027 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.305548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:15.306484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:20.107961 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:20.109084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:20.151016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:20.151405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.152280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:20.152280 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.152280 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:20.152280 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.152280 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.152280 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:20.152280 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.152280 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.153098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.153626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:20.233286 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:20.234406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:20.234713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:20.234930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.235698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:20.235698 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.235698 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:20.235698 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.235698 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.235698 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:20.235698 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.235698 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.237348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.238127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:20.238842 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:20.238842 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:20.238842 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:20.239144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.239899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:20.239899 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.239899 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:20.239899 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.239899 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.239899 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:20.239899 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.239899 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:20.241497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.242249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:20.242679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.243173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.243671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:20.243671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.243671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:20.243671 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:20.243794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:20.244167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:20.244430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.244918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:20.244918 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.244918 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:20.244918 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.244918 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.244918 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:20.244918 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.244918 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:20.246169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:20.246629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:20.246629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.246629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:20.246629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.246629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.246629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:20.246629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.246629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:20.248303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.248781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:20.249304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.249304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:20.302147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:20.302736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:20.303659 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:20.304065 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:20.304318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.305373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:20.305373 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.305373 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:20.305373 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.305373 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.305373 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:20.305373 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.305373 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.307127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:20.307913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:25.108749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:25.110176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:25.146248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:25.146694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.147712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:25.147712 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.147712 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:25.147712 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.147712 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.147712 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:25.147712 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.147712 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.148823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.150120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:25.234128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:25.235421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:25.235690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:25.235910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.236644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:25.236644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.236644 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:25.236644 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.236644 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.236644 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:25.236644 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.236644 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.238320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.239124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:25.240122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:25.240122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:25.240122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:25.240122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.240556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:25.240556 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.240556 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:25.240556 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.240556 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.240556 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:25.240556 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.240556 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:25.241704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.242504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:25.243231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.244195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.245057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:25.245057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.245057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:25.245057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:25.245057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:25.245057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:25.245369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.245827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:25.245827 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.245827 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:25.245827 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.245827 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.245827 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:25.245827 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.245827 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:25.246567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:25.247474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:25.247474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.247474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:25.247474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.247474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.247474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:25.247474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.247474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:25.247695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.248153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:25.248632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.248632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:25.303639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:25.303639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:25.304230 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:25.304600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:25.304819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.305798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:25.305798 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.305798 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:25.305798 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.305798 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.305798 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:25.305798 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.305798 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.307371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:25.308204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:30.109449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:30.111030 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:30.113008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:20:30.145797 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:30.146113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.146811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:30.146811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.146811 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:30.146811 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.146811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.146811 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:30.146811 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.146811 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.148042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.149103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:30.234837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:30.236202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:30.236528 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:30.236821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.237671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:30.237671 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.237671 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:30.237671 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.237671 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.237671 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:30.237671 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.237671 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.239540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.240326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:30.241104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:30.241104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:30.241104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:30.241104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.241808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:30.241808 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.241808 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:30.241808 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.241808 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.241808 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:30.241808 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.241808 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:30.243143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.243762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:30.244750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.245225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.245784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:30.245784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.245784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:30.245784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:30.245784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:30.246102 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:30.246329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.246945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:30.246945 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.246945 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:30.246945 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.246945 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.246945 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:30.246945 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.246945 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:30.248248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:30.248730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:30.248730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.248730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:30.248730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.248730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.248730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:30.248730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.248730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:30.250133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.250697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:30.251389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.251389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:30.303691 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:30.304659 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:30.305232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:30.305594 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:30.305828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.306721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:30.306721 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.306721 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:30.306721 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.306721 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.306721 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:30.306721 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.306721 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.308465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:30.309250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:35.110148 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:35.111829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:35.146296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:35.146682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.147458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:35.147458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.147458 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:35.147458 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.147458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.147458 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:35.147458 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.147458 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.148391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.149338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:35.235543 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:35.237020 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:35.237277 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:35.237480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.238361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:35.238361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.238361 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:35.238361 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.238361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.238361 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:35.238361 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.238361 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.239800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.240650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:35.241457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:35.241457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:35.241457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:35.241644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.242318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:35.242318 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.242318 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:35.242318 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.242318 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.242318 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:35.242318 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.242318 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:35.243576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.244339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:35.245208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.246119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.246918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:35.246918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.246918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:35.246918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:35.246918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:35.246918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:35.247265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.247550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:35.247550 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.247550 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:35.247550 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.247550 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.247550 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:35.247550 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.247550 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:35.248587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:35.249043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:35.249043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.249043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:35.249043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.249043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.249043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:35.249043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.249043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:35.249764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.250252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:35.250853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.250853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:35.304250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:35.304970 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:35.305582 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:35.305959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:35.306248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.307134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:35.307134 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.307134 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:35.307134 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.307134 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.307134 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:35.307134 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.307134 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.308911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:35.309705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:40.110829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:40.112494 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:40.153063 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:40.153640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.154626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:40.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:40.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.154626 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:40.154626 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.154626 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.155528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.156070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:40.236523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:40.238729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:40.238901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:40.239207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.240901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.241639 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.242472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.243445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:40.244191 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:40.244191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.244682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:40.244682 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.244682 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:40.244682 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.244682 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.244682 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:40.244682 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.244682 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:40.245459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:40.245999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.246472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:40.247060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:40.247540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:40.247540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.248588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:40.248588 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.248588 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:40.248588 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.248588 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.248588 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:40.248588 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.248588 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:40.249398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:40.249843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:40.249843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.249843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:40.249843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.249843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.249843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:40.249843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.249843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:40.250404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.250875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:40.251414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.251414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:40.304839 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:40.305493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:40.305899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:40.306194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:40.306341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.306962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:40.306962 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.306962 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:40.306962 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.306962 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.306962 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:40.306962 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.306962 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.308070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:40.308941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:45.111492 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:45.113084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:45.143508 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:45.143853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.144634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:45.144634 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.144634 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:45.144634 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.144634 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.144634 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:45.144634 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.144634 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.145387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.145960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:45.237197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:45.238500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:45.238789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:45.239017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.239793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:45.239793 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.239793 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:45.239793 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.239793 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.239793 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:45.239793 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.239793 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.241368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.242513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:45.243267 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:45.243267 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:45.243267 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:45.243267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.243651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:45.243651 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.243651 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:45.243651 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.243651 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.243651 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:45.243651 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.243651 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:45.244788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.245640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:45.246513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.247135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.247667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:45.247667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.247667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:45.247667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:45.247667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:45.247667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:45.247884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.248463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:45.248463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.248463 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:45.248463 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.248463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.248463 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:45.248463 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.248463 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:45.249694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:45.250348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:45.250348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.250348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:45.250348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.250348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.250348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:45.250348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.250348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:45.251286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.251776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:45.252393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.252393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:45.305471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:45.305992 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:45.306400 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:45.306733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:45.306946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.307692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:45.307692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.307692 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:45.307692 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.307692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.307692 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:45.307692 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.307692 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.309175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:45.309953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:50.112211 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:50.113965 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:50.148400 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:50.148591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.149276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:50.149276 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.149276 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:50.149276 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.149276 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.149276 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:50.149276 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.149276 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.151066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.152890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:50.237797 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:50.239379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:50.239670 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:50.239837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.240781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:50.240781 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.240781 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:50.240781 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.240781 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.240781 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:50.240781 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.240781 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.242771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.243565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:50.244354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:50.244354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:50.244354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:50.244354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.244631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:50.244631 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.244631 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:50.244631 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.244631 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.244631 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:50.244631 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.244631 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:50.245500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.246311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:50.246745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.247460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.248283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:50.248283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.248283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:50.248283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:50.248283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:50.248283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:50.248625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.248911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:50.248911 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.248911 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:50.248911 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.248911 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.248911 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:50.248911 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.248911 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:50.250167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:50.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:50.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:50.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:50.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:50.251723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.252447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:50.252992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.252992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:50.306127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:50.306751 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:50.307187 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:50.307501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:50.307713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.308438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:50.308438 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.308438 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:50.308438 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.308438 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.308438 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:50.308438 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.308438 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.310256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:50.311087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:55.113006 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:55.114918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:55.152284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:55.152622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.153820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:20:55.153820 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.153820 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:55.153820 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.153820 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.153820 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:55.153820 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.153820 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.154655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.157647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:55.238437 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:55.240152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:55.240441 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:55.240682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.242022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:55.242022 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.242022 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:55.242022 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.242022 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.242022 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:55.242022 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.242022 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.243257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.244021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:55.244923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:55.244923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:20:55.244923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:55.244923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.245241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:20:55.245241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.245241 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:55.245241 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.245241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.245241 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:55.245241 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.245241 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:55.246331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.247153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:20:55.247767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.248651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.249223 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:55.249564 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.249564 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:20:55.249781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:55.250286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:20:55.250286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.250286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:20:55.250286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.250286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.250286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:20:55.250286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.250286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:20:55.250799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.251315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:55.251914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.251914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:20:55.306676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:20:55.307119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:20:55.307798 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:55.308238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:20:55.308450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.309463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:20:55.309463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.309463 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:20:55.309463 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.309463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.309463 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:20:55.309463 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.309463 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.311048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:20:55.311895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:00.113741 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:00.115181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:00.144369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:00.144687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.145438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:00.145438 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.145438 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:00.145438 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.145438 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.145438 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:00.145438 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.145438 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.146627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.147315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:00.239350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:00.240599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:00.241016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:00.241325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.242351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:00.242351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.242351 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:00.242351 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.242351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.242351 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:00.242351 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.242351 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.243511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.244389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:00.245521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:00.245521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:00.245521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:00.245521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.246322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:00.246322 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.246322 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:00.246322 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.246322 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.246322 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:00.246322 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.246322 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:00.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.248220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:00.249459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.250105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.250630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:00.250630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.250630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:00.250630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:00.250630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:00.250805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:00.251079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.251543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:00.251543 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.251543 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:00.251543 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.251543 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.251543 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:00.251543 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.251543 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:00.252730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:00.253290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:00.253290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.253290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:00.253290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.253290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.253290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:00.253290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.253290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:00.254355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.254859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:00.255436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.255436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:00.307416 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:00.308010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:00.308590 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:00.308943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:00.309177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.309809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:00.309809 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.309809 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:00.309809 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.309809 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.309809 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:00.309809 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.309809 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.311182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:00.311955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:05.114374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:05.115889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:05.146758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:05.147323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.148341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:05.148341 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.148341 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:05.148341 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.148341 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.148341 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:05.148341 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.148341 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.149615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.150574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:05.239933 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:05.241299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:05.241680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:05.241934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.242837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:05.242837 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.242837 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:05.242837 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.242837 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.242837 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:05.242837 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.242837 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.244011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.245004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:05.245787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:05.245787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:05.245787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:05.245787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.246069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:05.246069 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.246069 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:05.246069 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.246069 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.246069 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:05.246069 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.246069 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:05.246994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.247665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:05.248133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.248853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.249760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:05.249760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.249760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:05.250185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:05.250455 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:05.250833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:05.251283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.251522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:05.251522 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.251522 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:05.251522 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.251522 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.251522 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:05.251522 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.251522 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:05.252352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:05.252785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:05.252785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.252785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:05.252785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.252785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.252785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:05.252785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.252785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:05.253564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:05.254490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.254490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:05.307875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:05.308511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:05.309090 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:05.309518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:05.309789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.310585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:05.310585 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.310585 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:05.310585 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.310585 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.310585 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:05.310585 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.310585 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.311993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:05.312713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:10.115317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:10.116244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:10.141950 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:10.142338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.143200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:10.143200 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.143200 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:10.143200 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.143200 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.143200 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:10.143200 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.143200 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.144062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.144821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:10.240627 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:10.241861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:10.242075 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:10.242177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.242822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:10.242822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.242822 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:10.242822 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.242822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.242822 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:10.242822 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.242822 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.244599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.245442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:10.246201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:10.246201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:10.246201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:10.246201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.246422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:10.246422 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.246422 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:10.246422 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.246422 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.246422 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:10.246422 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.246422 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:10.247814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.248422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:10.249366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.249815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.250341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:10.250341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.250341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:10.250576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:10.251179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:10.251474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:10.251636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.252089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:10.252089 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.252089 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:10.252089 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.252089 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.252089 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:10.252089 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.252089 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:10.253336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:10.253751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:10.253751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.253751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:10.253751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.253751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.253751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:10.253751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.253751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:10.255064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.255724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:10.256317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.256317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:10.308647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:10.309216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:10.309532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:10.309692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:10.309777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.310349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:10.310349 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.310349 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:10.310349 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.310349 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.310349 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:10.310349 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.310349 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.312332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:10.313240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:15.115829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:15.117388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:15.156542 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:15.157091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.158279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:15.158279 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.158279 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:15.158279 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.158279 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.158279 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:15.158279 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.158279 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.159440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.160313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:15.241383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:15.242683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:15.243160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:15.243410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.244287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:15.244287 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.244287 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:15.244287 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.244287 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.244287 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:15.244287 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.244287 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.245960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.246940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:15.247729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:15.247729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:15.247729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:15.247915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.248563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:15.248563 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.248563 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:15.248563 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.248563 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.248563 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:15.248563 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.248563 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:15.249814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.250594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:15.251405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.252192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.253037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:15.253037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.253037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:15.253037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:15.253037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:15.253037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:15.253037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.253382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:15.253382 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.253382 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:15.253382 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.253382 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.253382 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:15.253382 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.253382 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:15.254602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:15.255364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:15.255364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.255364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:15.255364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.255364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.255364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:15.255364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.255364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:15.255924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.256448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:15.256990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.256990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:15.309174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:15.309859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:15.310702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:15.311281 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:15.311643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.312672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:15.312672 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.312672 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:15.312672 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.312672 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.312672 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:15.312672 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.312672 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.314377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:15.315598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:20.116490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:20.118752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:20.153346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:20.153714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.154507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:20.154507 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.154507 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:20.154507 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.154507 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.154507 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:20.154507 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.154507 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.155536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.156309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:20.242004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:20.243246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:20.243587 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:20.243824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.244647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:20.244647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.244647 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:20.244647 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.244647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.244647 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:20.244647 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.244647 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.246315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.246888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:20.247437 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:20.247437 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:20.247516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:20.247713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.248212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:20.248212 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.248212 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:20.248212 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.248212 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.248212 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:20.248212 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.248212 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:20.248960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.249428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:20.249837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.250328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:20.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.250844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:20.252146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:20.252676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:20.253074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:20.253294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.253824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:20.253824 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.253824 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:20.253824 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.253824 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.253824 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:20.253824 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.253824 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:20.255265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:20.255745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:20.255745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.255745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:20.255745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.255745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.255745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:20.255745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.255745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:20.256362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.256824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:20.257391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.257391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:20.309826 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:20.310374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:20.310964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:20.311355 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:20.311579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.312349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:20.312349 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.312349 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:20.312349 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.312349 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.312349 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:20.312349 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.312349 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.313359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:20.313849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:25.117409 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:25.118290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:25.150327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:25.150689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.151544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:25.151544 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.151544 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:25.151544 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.151544 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.151544 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:25.151544 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.151544 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.152732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.153489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:25.242558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:25.244280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:25.244677 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:25.244965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.245968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:25.245968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.245968 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:25.245968 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.245968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.245968 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:25.245968 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.245968 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.247644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.248680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:25.249252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:25.249252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:25.250254 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:25.250577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.251492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:25.251492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.251492 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:25.251492 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.251492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.251492 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:25.251492 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.251492 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:25.252754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.253240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:25.254263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.254766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.255296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:25.255296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.255296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:25.255296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:25.255296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:25.255593 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:25.255837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.256458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:25.256458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.256458 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:25.256458 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.256458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.256458 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:25.256458 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.256458 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:25.257403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:25.258059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:25.258059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.258059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:25.258059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.258059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.258059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:25.258059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.258059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:25.258507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.259050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:25.259585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.259585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:25.310413 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:25.311154 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:25.311899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:25.312517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:25.312627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.313670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:25.313670 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.313670 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:25.313670 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.313670 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.313670 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:25.313670 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.313670 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.315241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:25.315749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:30.118055 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:30.119137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:30.121646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:21:30.159836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:30.160198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.160667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:30.160667 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.160667 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:30.160667 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.160667 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.160667 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:30.160667 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.160667 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.162135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.162967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:30.243099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:30.244077 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:30.244314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:30.244441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.244969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:30.244969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.244969 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:30.244969 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.244969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.244969 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:30.244969 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.244969 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.245861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.246395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:30.248547 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:30.249064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:30.249467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:30.249616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.250078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:30.250078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.250078 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:30.250078 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.250078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.250078 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:30.250078 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.250078 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:30.250809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.251320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:30.252175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.252777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.253451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:30.253451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.253451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:30.254955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:30.255398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:30.255701 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:30.255862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.257873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:30.257873 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.257873 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:30.257873 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.257873 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.257873 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:30.257873 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.257873 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:30.258631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:30.259141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:30.259141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.259141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:30.259141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.259141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.259141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:30.259141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.259141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:30.259511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.260082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:30.260885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.260885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:30.310984 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:30.311218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:30.311773 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:30.311994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:30.312115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.312591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:30.312591 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.312591 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:30.312591 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.312591 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.312591 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:30.312591 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.312591 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.313594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:30.314167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:35.118478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:35.119662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:35.153085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:35.153326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.153906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:35.153906 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.153906 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:35.153906 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.153906 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.153906 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:35.153906 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.153906 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.155302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.156777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:35.243589 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:35.244600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:35.244838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:35.245003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.245651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:35.245651 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.245651 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:35.245651 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.245651 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.245651 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:35.245651 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.245651 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.246500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.247663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:35.248901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:35.249501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:35.249935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:35.250153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.250755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:35.250755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.250755 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:35.250755 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.250755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.250755 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:35.250755 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.250755 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:35.252104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.252528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:35.253515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.254117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.254831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:35.254831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.254831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:35.255313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:35.255923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:35.256184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:35.256413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.256850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:35.256850 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.256850 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:35.256850 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.256850 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.256850 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:35.256850 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.256850 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:35.259217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.259881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:35.260595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.261406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:35.311398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:35.311915 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:35.312328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:35.312608 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:35.312746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.313325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:35.313325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.313325 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:35.313325 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.313325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.313325 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:35.313325 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.313325 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.315482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:35.316376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:40.119251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:40.121691 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:40.156408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:40.156805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.157741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:40.157741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.157741 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:40.157741 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.157741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.157741 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:40.157741 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.157741 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.158724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.159720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:40.244278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:40.245809 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:40.246221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:40.246499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:40.247641 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.247641 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:40.247641 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.247641 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.247641 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:40.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.247641 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.249777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.251072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:40.251675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:40.251675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:40.251675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:40.251675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.252181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:40.252181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.252181 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:40.252181 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.252181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.252181 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:40.252181 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.252181 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:40.253112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:40.253886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:40.253886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.253886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:40.253886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.253886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.253886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:40.253886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.253886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:40.254494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.255065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.255690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:40.255690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.255690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:40.255690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:40.255923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:40.256200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:40.256471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.257305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:40.257305 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.257305 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:40.257305 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.257305 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.257305 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:40.257305 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.257305 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:40.258114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:40.258599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:40.258599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.258599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:40.258599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.258599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.258599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:40.258599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.258599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:40.259226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.259732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:40.260454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.260454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:40.312134 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:40.312696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:40.313272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:40.313765 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:40.313966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.314783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:40.314783 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.314783 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:40.314783 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.314783 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.314783 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:40.314783 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.314783 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.315926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:40.316794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:45.120694 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:45.244873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:45.246273 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:45.250440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:45.251738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:45.252336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:45.252496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:45.254010 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.254010 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:45.254010 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.254010 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.254010 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:45.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.254010 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:45.256099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.256892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:45.257711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:45.257711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:45.257711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:45.257711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.258315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:45.258612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:45.258612 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.258612 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:45.258612 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.258612 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.258612 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:45.258612 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.258612 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:45.260002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:45.260704 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:45.263755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.264276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.264844 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:45.266399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.266870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:45.268217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.268769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:45.269294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.269294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:45.312636 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:45.313377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:45.314008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:45.314411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:45.314598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.315391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:21:45.315391 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.315391 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:45.315391 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.315391 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.315391 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:45.315391 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.315391 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.317132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.317635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:45.545902 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:45.546530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.547323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:21:45.547323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.547323 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:45.547323 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.547323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.547323 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:45.547323 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.547323 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:45.549130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:50.245537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:50.247017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:50.254222 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:50.257194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:50.257683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:50.261084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:50.261897 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:50.262991 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:50.263486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:50.263647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.267629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:50.267629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.267629 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:50.267629 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.267629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.267629 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:50.267629 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.267629 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:50.270321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.271404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:50.272356 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:50.272356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.272463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:21:50.272463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.272463 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:50.272463 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.272463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.272463 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:50.272463 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.272463 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:50.275637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.276506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:50.279117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:50.281688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.282517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.283379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:50.283379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.283379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:50.283379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.284797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:50.284797 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.284797 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:50.284797 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.284797 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.284797 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:50.284797 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.284797 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:50.288297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.288807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:50.291087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.291608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:50.292346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.292346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:50.305394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:50.305836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.306728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:50.306728 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.306728 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:50.306728 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.306728 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.306728 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:50.306728 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.306728 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.307740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.308475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:50.313265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:50.313887 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:50.314340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:50.314705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:50.314705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.315323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:50.315323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.315323 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:50.315323 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.315323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.315323 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:50.315323 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.315323 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.317373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:50.317710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:55.246203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:55.247855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:55.257013 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:55.258181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:55.260362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:55.261651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:21:55.262051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:55.262460 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:55.262704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.263540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:55.263787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.264434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:21:55.264434 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.264434 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:55.264434 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.264434 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.264434 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:55.264434 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.264434 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:55.267882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:21:55.267882 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.267882 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:55.267882 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.267882 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.267882 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:55.267882 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.267882 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:55.271596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.272612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:55.273836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.274434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:55.275535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:21:55.279086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.280053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.280991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:55.280991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.280991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:55.280991 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:55.280991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.281792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:21:55.281792 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.281792 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:55.281792 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.281792 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.281792 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:55.281792 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.281792 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:21:55.284767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.285198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:21:55.288730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.289217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:55.289676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.289676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:55.299849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:55.300169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.300758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:21:55.300758 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.300758 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:55.300758 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.300758 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.300758 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:55.300758 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.300758 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.301911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.302815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:21:55.313837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:21:55.314304 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:21:55.314807 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:55.315300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:21:55.315693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.316480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:21:55.316480 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.316480 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:21:55.316480 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.316480 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.316480 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:21:55.316480 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.316480 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.318104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:21:55.319221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:00.247227 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:00.248698 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:00.259225 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:00.260535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:00.260822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:00.261097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.262265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:00.263002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:00.263002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.263002 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:00.263002 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.263002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.263002 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:00.263002 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.263002 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:00.264479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.265080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:00.265670 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:00.265670 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:00.265761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:00.266245 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:00.266658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.270938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:00.270938 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.270938 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:00.270938 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.270938 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.270938 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:00.270938 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.270938 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:00.272244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.273546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:00.274733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.275723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.276539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:00.276539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.276539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:00.276539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:00.276539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.277238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:00.277238 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.277238 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:00.277238 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.277238 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.277238 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:00.277238 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.277238 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:00.279539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.279859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:00.283170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.283530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:00.283896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.283896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:00.305211 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:00.305576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.306290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:00.306290 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.306290 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:00.306290 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.306290 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.306290 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:00.306290 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.306290 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.307235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.308058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:00.314394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:00.314813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:00.315325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:00.315664 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:00.315879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.316649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:00.316649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.316649 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:00.316649 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.316649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.316649 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:00.316649 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.316649 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.318455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:00.318966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:05.248465 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:05.249902 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:05.262003 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:05.263209 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:05.263694 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:05.265308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:05.266277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.268718 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:05.269270 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:05.270628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:05.270628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.270628 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:05.270628 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.270628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.270628 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:05.270628 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.270628 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:05.271680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:05.272081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:05.275280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.275814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:05.276453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.278070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:05.278070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.278070 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:05.278070 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.278070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.278070 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:05.278070 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.278070 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:05.279994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.280566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:05.280831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:05.284942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.285421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.285910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:05.285910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.285910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:05.286235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.287139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:05.287139 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.287139 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:05.287139 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.287139 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.287139 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:05.287139 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.287139 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:05.289762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.290215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:05.291694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.292145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:05.292642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.292642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:05.302519 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:05.302738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.303361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:05.303361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.303361 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:05.303361 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.303361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.303361 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:05.303361 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.303361 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.304202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.304672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:05.315203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:05.315733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:05.316458 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:05.316971 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:05.317356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.318360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:05.318360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.318360 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:05.318360 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.318360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.318360 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:05.318360 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.318360 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.320286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:05.320807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:10.248969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:10.250442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:10.265214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:10.266290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:10.267490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:10.268366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:10.268482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.269085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:10.269085 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.269085 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:10.269085 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.269085 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.269085 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:10.269085 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.269085 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:10.271634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.272631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:10.274074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:10.274074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:10.274395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:10.274623 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:10.274868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.277001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:10.277001 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.277001 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:10.277001 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.277001 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.277001 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:10.277001 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.277001 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:10.278368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.279017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:10.280649 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:10.280649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:10.283499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.284258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.285761 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:10.287854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.288430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:10.292498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.293006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:10.293495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.293495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:10.296444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:10.296665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.297273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:10.297273 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.297273 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:10.297273 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.297273 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.297273 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:10.297273 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.297273 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.298145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.298605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:10.315945 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:10.316373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:10.316835 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:10.317292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:10.317536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.318257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:10.318257 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.318257 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:10.318257 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.318257 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.318257 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:10.318257 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.318257 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.319401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:10.320058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:15.249703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:15.252011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:15.266873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:15.268699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:15.268890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:15.269030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.270382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:15.270382 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.270382 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:15.270382 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.270382 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.270382 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:15.270382 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.270382 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:15.271826 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:15.276318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.277129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:15.278089 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:15.279693 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:15.281232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:15.282508 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:15.282681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.283241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:15.283241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.283241 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:15.283241 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.283241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.283241 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:15.283241 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.283241 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:15.290021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:15.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.291240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:15.291863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.294077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:15.294077 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.294077 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:15.294077 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.294077 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.294077 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:15.294077 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.294077 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:15.296372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.297018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:15.298422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.298797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:15.299189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.299189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:15.301740 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:15.301994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.302582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:15.302582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.302582 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:15.302582 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.302582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.302582 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:15.302582 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.302582 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.303694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.304314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:15.316362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:15.317197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:15.317539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:15.317894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:15.318167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.318845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:15.318845 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.318845 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:15.318845 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.318845 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.318845 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:15.318845 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.318845 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.319968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:15.320666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:20.250378 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:20.251868 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:20.269444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:20.271727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:20.271890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:20.272084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.272779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:20.272779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.272779 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:20.272779 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.272779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.272779 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:20.272779 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.272779 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:20.277874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.278524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:20.279204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:20.279460 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:20.279821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:20.281845 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:20.284807 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:20.285033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.285652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:20.285901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.287617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:20.287617 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.287617 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:20.287617 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.287617 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.287617 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:20.287617 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.287617 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:20.290117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.290655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:20.292902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:20.292902 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.292902 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:20.292902 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.292902 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.292902 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:20.292902 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.292902 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:20.294968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.295510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.296045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:20.296045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.296045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:20.301642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.302246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:20.302501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:20.304572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.305216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.306260 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.306812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:20.316835 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:20.317510 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:20.318265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:20.319113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:20.319113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.319800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:20.319800 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.319800 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:20.319800 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.319800 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.319800 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:20.319800 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.319800 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.321454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:20.322344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:25.251022 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:25.252810 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:25.271421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:25.273542 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:25.274656 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:25.274795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.277659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:25.277659 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.277659 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:25.277659 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.277659 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.277659 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:25.277659 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.277659 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:25.284216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.285447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:25.286824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:25.286824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:25.286824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:25.288624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:25.291792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:25.292386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.293317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:25.293317 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.293317 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:25.293317 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.293317 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.293317 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:25.293317 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.293317 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:25.294277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:25.294766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:25.295628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.296136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.296626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:25.296626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.296626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:25.296626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.296626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:25.296794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.297250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:25.297250 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.297250 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:25.297250 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.297250 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.297250 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:25.297250 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.297250 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:25.297972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.298438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:25.299117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.299579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.300119 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.300804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.301292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:25.317434 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:25.317881 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:25.318242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:25.318444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:25.318585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.319090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:25.319090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.319090 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:25.319090 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.319090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.319090 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:25.319090 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.319090 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.320792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:25.321572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:30.251879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:30.253038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:30.254468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:22:30.272080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:30.273577 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:30.273681 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:30.273869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.277043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:30.277043 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.277043 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:30.277043 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.277043 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.277043 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:30.277043 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.277043 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:30.277794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.278771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:30.289960 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:30.290255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:30.290838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:30.292702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:30.293317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:30.293669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.295702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:30.296128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.296999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:30.296999 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.296999 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:30.296999 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.296999 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.296999 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:30.296999 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.296999 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:30.299576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.300355 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:30.302126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.303090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:30.303837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:30.304823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:30.305816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.306894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:30.307615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:30.308413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.309276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:30.317831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:30.318347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:30.318751 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:30.319132 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:30.319359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.320059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:30.320059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.320059 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:30.320059 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.320059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.320059 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:30.320059 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.320059 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.321646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:30.322450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:35.252489 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:35.254122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:35.273187 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:35.275891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:35.277175 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:35.277355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.279367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:35.279367 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.279367 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:35.279367 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.279367 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.279367 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:35.279367 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.279367 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:35.283406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.284547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:35.291596 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:35.292954 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:35.293562 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:35.294090 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:35.295083 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:35.295572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.296483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:35.296483 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.296483 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:35.296483 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.296483 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.296483 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:35.296483 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.296483 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:35.297558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.298185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:35.299168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.301472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.302019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:35.302019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.302019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:35.302019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:35.302019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.302414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:35.302414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.302414 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:35.302414 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.302414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.302414 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:35.302414 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.302414 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:35.303730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.304156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:35.304910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.305406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:35.305872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.305872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:35.305999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:35.306504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.307251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:35.307251 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.307251 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:35.307251 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.307251 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.307251 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:35.307251 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.307251 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.308863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.309352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:35.318329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:35.318662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:35.319190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:35.319568 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:35.319788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.320476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:35.320476 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.320476 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:35.320476 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.320476 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.320476 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:35.320476 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.320476 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.321561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:35.322443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:40.253394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:40.254591 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:40.275496 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:40.277372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:40.278761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:40.279206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.282230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:40.282230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.282230 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:40.282230 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.282230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.282230 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:40.282230 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.282230 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:40.285826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.286803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:40.293749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:40.293948 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:40.294471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:40.296682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:40.297177 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:40.297517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.298540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:40.298540 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.298540 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:40.298540 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.298540 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.298540 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:40.298540 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.298540 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:40.299431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:40.301159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:40.302457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:40.304554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.305567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:40.306409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.307196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:40.308069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:40.308797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.309690 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.310419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.311292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:40.318875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:40.319418 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:40.320060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:40.320493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:40.320751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.321450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:40.321450 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.321450 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:40.321450 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.321450 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.321450 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:40.321450 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.321450 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.322834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:40.323378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:45.254226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:45.256164 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:45.276178 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:45.278332 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:45.279433 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:45.279433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.280372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:45.280372 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.280372 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:45.280372 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.280372 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.280372 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:45.280372 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.280372 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:45.282193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.283037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:45.292960 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:45.293350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.294151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:45.294151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.294151 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:45.294151 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.294151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.294151 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:45.294151 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.294151 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.295448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.296224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:45.296973 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:45.296973 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:45.296973 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:45.297474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:45.298196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:45.298467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.299284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:45.299284 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.299284 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:45.299284 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.299284 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.299284 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:45.299284 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.299284 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:45.300959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:45.301428 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:45.303347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.303796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.304352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:45.304352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.304352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:45.304352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.304574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:45.304574 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.304574 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:45.304574 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.304574 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.304574 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:45.304574 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.304574 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:45.305703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.306134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:45.307024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.307459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:45.307969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.307969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:45.319360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:45.319614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:45.319970 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:45.320335 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:45.320554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.321204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:45.321204 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.321204 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:45.321204 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.321204 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.321204 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:45.321204 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.321204 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.322038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:45.322807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:50.254915 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:50.256848 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:50.276727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:50.279421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:50.279639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:50.279802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.280524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:50.280524 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.280524 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:50.280524 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.280524 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.280524 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:50.280524 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.280524 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:50.281934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.282930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:50.293817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:50.294037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.294560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:22:50.294560 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.294560 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:50.294560 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.294560 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.294560 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:50.294560 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.294560 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.295373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.295887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:50.296669 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:50.297128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:50.297553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:50.297962 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:50.298275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:50.298426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.298865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:22:50.298865 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.298865 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:50.298865 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.298865 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.298865 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:50.298865 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.298865 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:50.299685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:50.300149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:50.301086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.302000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:50.302794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.303578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.304248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:50.304897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.305722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.305722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:50.305722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.305722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:50.319797 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:50.320223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:50.320493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:50.320681 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:50.320787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.321181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:50.321181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.321181 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:50.321181 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.321181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.321181 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:50.321181 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.321181 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.321927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:50.322720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:55.255640 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:55.257184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:55.433404 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:55.433638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:55.433750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:55.433927 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:22:55.434996 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:22:55.441469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:55.444150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:55.444751 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:22:55.445061 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:55.445798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.447700 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:55.448062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.449109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:22:55.449109 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.449109 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:55.449109 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.449109 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.449109 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:55.449109 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.449109 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.451706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.452745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:55.453625 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:55.454011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:22:55.454011 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.454011 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:55.454011 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.454011 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.454011 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:55.454011 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.454011 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:55.454462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:55.457882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.458519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:55.458783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:22:55.458783 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.458783 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:55.458783 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.458783 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.458783 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:55.458783 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.458783 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:55.461609 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:22:55.465422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.465906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.466446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:55.466446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.466446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:22:55.469371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.470155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:22:55.470957 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:22:55.473803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.474618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:55.475434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.475434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:55.475434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.475919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:22:55.475919 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.475919 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:55.475919 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.475919 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.475919 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:55.475919 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.475919 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.478755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.479303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:22:55.490308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:22:55.490636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.491388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:22:55.491388 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.491388 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:22:55.491388 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.491388 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.491388 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:22:55.491388 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.491388 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:22:55.493285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:00.256372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:00.258223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:00.291115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:00.291318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.291901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:00.291901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.291901 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:00.291901 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.291901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.291901 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:00.291901 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.291901 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.293619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.293945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:00.434531 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:00.435322 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:00.435774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:00.437086 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:00.437390 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:00.437639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.438608 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:00.438885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.439733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:23:00.439733 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.439733 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:00.439733 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.439733 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.439733 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:00.439733 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.439733 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:00.440955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.441509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:00.442433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.442850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:00.443439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.445161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.445681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:00.446201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:00.446201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:00.446555 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:00.447340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:00.447584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.448162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:00.448162 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.448162 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:00.448162 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.448162 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.448162 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:00.448162 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.448162 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:00.449310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:00.449781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:00.450964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.451438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:00.452002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.452002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:00.452002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:00.452002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.452236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:23:00.452236 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.452236 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:00.452236 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.452236 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.452236 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:00.452236 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.452236 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.453307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:00.453951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:05.257064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:05.258586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:05.292205 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:05.292560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.293402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:05.293402 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.293402 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:05.293402 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.293402 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.293402 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:05.293402 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.293402 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.294559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.295565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:05.451551 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:05.451551 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:05.451916 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:05.452758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:05.452892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:05.453766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:05.454298 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:05.454564 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:05.454750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:05.454957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.455652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:05.455652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.455652 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:05.455652 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.455652 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.455652 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:05.455652 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.455652 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.456674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.457058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:05.457414 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:05.457414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.457462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:05.457613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.457955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:05.457955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.457955 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:05.457955 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.457955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.457955 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:05.457955 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.457955 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:05.459191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:05.459572 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:05.459842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:05.459842 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.459842 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:05.459842 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.459842 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.459842 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:05.459842 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.459842 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:05.460409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:05.460827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:05.462126 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.462606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:05.463132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:05.463629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.464169 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.464762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:05.465383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:10.257697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:10.259473 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:10.296375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:10.296650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.297781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:10.297781 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.297781 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:10.297781 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.297781 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.297781 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:10.297781 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.297781 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.298844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.299532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:10.435966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:10.437250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:10.437590 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:10.437820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.438649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:10.438649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.438649 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:10.438649 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.438649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.438649 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:10.438649 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.438649 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.439952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.440968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:10.451921 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:10.452568 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:10.453150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:10.453463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.454311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:10.454549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:10.455242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:10.456309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:10.456309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.456309 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:10.456309 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.456309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.456309 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:10.456309 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.456309 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:10.456766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:10.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:10.458997 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:10.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.461265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.462080 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:10.463265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.463685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:10.465229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.465705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:10.466189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.466189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:10.466189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:10.466189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:10.466304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.467361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:10.467361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.467361 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:10.467361 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.467361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.467361 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:10.467361 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.467361 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.468971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:10.469683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:15.258397 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:15.260508 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:15.292505 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:15.292727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.293059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:15.293059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.293059 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:15.293059 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.293059 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.293059 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:15.293059 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.293059 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.293790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.294653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:15.436638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:15.437781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:15.438128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:15.438374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.439125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:15.439125 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.439125 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:15.439125 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.439125 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.439125 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:15.439125 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.439125 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.440330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.441202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:15.452534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:15.453000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:15.453750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:15.454059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.454779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:15.454779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.454779 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:15.454779 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.454779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.454779 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:15.454779 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.454779 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:15.455737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.456492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:15.456945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.458524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:15.458524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.458524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:15.458524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:15.458524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:15.458524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:15.458524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:15.458899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:15.459016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.459738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:15.459738 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.459738 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:15.459738 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.459738 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.459738 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:15.459738 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.459738 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:15.460754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:15.461185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:15.462505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.463008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:15.463530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.463530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:15.463530 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:15.463530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.463682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:15.463682 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.463682 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:15.463682 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.463682 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.463682 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:15.463682 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.463682 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.464663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:15.465161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:20.259017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:20.260237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:20.296694 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:20.296926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.297455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:20.297455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.297455 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:20.297455 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.297455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.297455 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:20.297455 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.297455 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.298595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.300028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:20.437518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:20.438920 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:20.439370 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:20.439627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.440462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:20.440462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.440462 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:20.440462 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.440462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.440462 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:20.440462 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.440462 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.442360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.443412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:20.453446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:20.453880 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:20.454457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:20.454725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.455504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:20.455504 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.455504 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:20.455504 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.455504 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.455504 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:20.455504 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.455504 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:20.456835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.457374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:20.458476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.458930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.459597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:20.459597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.459597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:20.459597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:20.459597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:20.459597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:20.459911 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:20.460540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:20.460928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.461627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:20.461627 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.461627 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:20.461627 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.461627 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.461627 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:20.461627 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.461627 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:20.462733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:20.463399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:20.463399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.463399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:20.463399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.463399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.463399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:20.463399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.463399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:20.464213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.464787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:20.465371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.465371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:20.465371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:20.465371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:20.465371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.465755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:20.465755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.465755 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:20.465755 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.465755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.465755 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:20.465755 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.465755 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.466832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:20.467379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:24.806874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:24.816622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 26 01:23:24.816772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105681) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}}
Mar 26 01:23:24.816772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:23:24.846067 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, resource={'id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'name': 'private', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['02d96a4b-4c93-4b95-ad0f-f11d085f0699', '0503b55a-1b69-49b5-b418-16ba37be90e9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T00:58:10Z', 'updated_at': '2026-03-26T00:58:13Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4449}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.846293 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.846611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6fe31530-ff9f-476b-aea1-4804059b76cd, external_ids_update=None, columns={'tag': 4449}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.846707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.847271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:24.847271 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:24.847271 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:24.847271 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:24.847271 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:24.847271 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:24.847271 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.847271 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:24.848440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:23:24.849118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:24.849855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:23:24.849855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.849855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:23:24.849855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:23:24.879008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-db438906-c74f-4dc8-883c-6e56962e551b, resource={'id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'name': 'public', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['1c3e4583-6e63-4953-8050-3341662a5ec6', '23682989-62bf-4e07-a74f-0cb01077d5ac'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-26T00:58:16Z', 'updated_at': '2026-03-26T00:58:25Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.879255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-db438906-c74f-4dc8-883c-6e56962e551b, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.879634 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-200618ec-0540-41b0-9429-dc7dfc74abb0, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.879832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.880423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:24.880423 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:24.880423 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:24.880423 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:24.880423 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:24.880423 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:24.880423 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.880423 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:24.881540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:23:24.882183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:24.882875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:23:24.882875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.882875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:23:24.882875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:23:24.908339 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, resource={'id': 'f7e819e4-7101-412b-b047-baa5e16b2a52', 'name': 'admin_net', 'tenant_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'project_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f2e900ee-4d88-4151-8e50-661745950632'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:14Z', 'updated_at': '2026-03-26T01:02:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4803}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.908632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.908847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d43d91f1-f24a-411f-bae7-af85048f95b9, external_ids_update=None, columns={'tag': 4803}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.908926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.909597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:24.909597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:24.909597 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:24.909597 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:24.909597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:24.909597 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:24.909597 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.909597 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:24.910636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:23:24.911197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:24.911809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:23:24.911809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.911809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:23:24.911809 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:23:24.944287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, resource={'id': 'f481c2f9-5052-46d5-821a-6c8712ec70ec', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['329ea1d3-f656-45b1-bf23-1d927d4578ea'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:50Z', 'updated_at': '2026-03-26T01:02:51Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19903}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.944956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.945400 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3aa5be7a-5c67-4f1a-bdf2-7bf6912e871d, external_ids_update=None, columns={'tag': 19903}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:24.945810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.946442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:24.946442 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:24.946442 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:24.946442 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:24.946442 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:24.946442 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:24.946442 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.946442 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:24.947656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:23:24.948361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:24.949027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:23:24.949027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:24.949027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:23:24.949027 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:23:25.020928 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, resource={'id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'enable_snat': True}, 'gw_port_id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-ryt6n', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'ip_address': '172.24.5.142', 'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac'}, {'ip_address': '2001:db8::16a', 'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 3, 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8'}, resource_type=routers, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.021432 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.021570 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.021781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.021957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.022797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:25.022797 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.022797 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:25.022797 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.022797 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.022797 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:25.022797 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.022797 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:25.023249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Unable to update router e4efd798-3c25-4ea9-bd45-fa7a05aeaef9. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 26 01:23:25.024620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:23:25.025425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:23:25.026129 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:23:25.129667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, resource={'id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'name': '', 'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'project_id': '', 'mac_address': 'fa:16:3e:4e:d4:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160849', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.130192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::16a/64', '172.24.5.142/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.130851 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=14a0aa42-40aa-4508-9af3-cffdffef9818, lrouter_port=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.131318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.132196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:23:25.132196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.132196 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:25.132196 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.132196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.132196 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:25.132196 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.132196 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:25.133054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.133382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.133779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:23:25.133779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.133779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:23:25.133779 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Synchronization completed (took 0.32 seconds)
Mar 26 01:23:25.259654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:25.261295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:25.298471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.298814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:23:25.299645 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.299645 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:25.299645 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.299645 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.299645 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:25.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.300351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.301301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:25.438415 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:25.439476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.439811 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.440118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.441175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:25.441175 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.441175 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:25.441175 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.441175 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.441175 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:25.441175 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.441175 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.443285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.444099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:25.453999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:25.454697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:25.454913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.455245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.455900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:25.455900 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.455900 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:25.455900 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.455900 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.455900 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:25.455900 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.455900 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:25.456555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:25.457526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.458295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.459158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:25.460062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:25.460259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:25.460850 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:25.461553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:25.462152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.462521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.463571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:23:25.463571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.463571 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:25.463571 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.463571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.463571 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:25.463571 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.463571 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:25.464906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:25.465580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:25.466625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.467225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:25.467728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.467728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:25.467728 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:25.467728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.467850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:23:25.467850 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.467850 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:25.467850 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.467850 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.467850 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:25.467850 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.467850 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.469764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:25.470377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:30.260329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:30.261521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:30.262830 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=434 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:23:30.291309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:30.291672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.292419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:23:30.292419 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.292419 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:30.292419 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.292419 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.292419 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:30.292419 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.292419 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.293662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.294754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:30.438999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:30.440527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:30.440527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:30.440527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.441100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:30.441100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.441100 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:30.441100 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.441100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.441100 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:30.441100 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.441100 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.442397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.442946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:30.454579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:30.454932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:30.455535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:30.455754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.456294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:30.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:30.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:30.456294 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.456294 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:30.457246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.457746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:30.459732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.460297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.460793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:30.460793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.460793 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:30.460793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:30.461307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:30.461713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:30.462232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:30.463004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:30.463222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.463787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:23:30.463787 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.463787 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:30.463787 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.463787 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.463787 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:30.463787 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.463787 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:30.464984 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:30.465430 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:30.466833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.467242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:30.467591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.467591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:30.467591 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:30.467591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.467815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:30.467815 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.467815 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:30.467815 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.467815 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.467815 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:30.467815 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.467815 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.469143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:30.469527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:35.261019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:35.262649 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:35.300008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:35.300435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.301384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:23:35.301384 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.301384 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:35.301384 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.301384 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.301384 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:35.301384 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.301384 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.302705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.303562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:35.439693 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:35.440845 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:35.441201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:35.441574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.442442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:35.442442 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.442442 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:35.442442 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.442442 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.442442 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:35.442442 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.442442 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.443554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.444114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:35.454987 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:35.455490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:35.455967 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:35.456261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.457032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:35.457032 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.457032 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:35.457032 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.457032 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.457032 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:35.457032 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.457032 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:35.458383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.458836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:35.459933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.460387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.460863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:35.460863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.460863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:35.461133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:35.461517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:35.461925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:35.462326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.462963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:23:35.462963 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.462963 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:35.462963 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.462963 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.462963 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:35.462963 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.462963 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:35.463857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:35.464281 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:35.466230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.466688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:35.467208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.467208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:35.467208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:35.467208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:35.467484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:35.467667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.468375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:35.468375 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.468375 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:35.468375 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.468375 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.468375 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:35.468375 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.468375 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.470283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:35.470773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:40.261756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:40.263148 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:40.289776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:40.290229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.291024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:23:40.291024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.291024 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:40.291024 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.291024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.291024 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:40.291024 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.291024 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.291907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.292661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:40.440476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:40.441669 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:40.441992 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:40.442232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.442991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:40.442991 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.442991 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:40.442991 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.442991 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.442991 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:40.442991 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.442991 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.444588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.445094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:40.455386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:40.455861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:40.456365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:40.456646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.457462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:40.457462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.457462 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:40.457462 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.457462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.457462 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:40.457462 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.457462 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:40.458659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.459350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:40.460249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.460995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.461795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:40.461795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.461795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:40.461795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:40.462001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:40.462439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:40.462768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.463493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:23:40.463493 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.463493 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:40.463493 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.463493 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.463493 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:40.463493 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.463493 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:40.464772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.465209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:40.466563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.467008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:40.467697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.467697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:40.467697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:40.467697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:40.467838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:40.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:40.468352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.468956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:40.468956 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.468956 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:40.468956 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.468956 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.468956 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:40.468956 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.468956 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.469815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:40.470271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:45.262520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:45.264478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:45.297384 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:45.297763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.298638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:23:45.298638 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.298638 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:45.298638 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.298638 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.298638 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:45.298638 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.298638 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.299852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.301345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:45.441240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:45.442453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:45.442788 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:45.443056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.443864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:45.443864 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.443864 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:45.443864 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.443864 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.443864 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:45.443864 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.443864 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.445584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.446652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:45.455868 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:45.456345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:45.456882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:45.457211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.457968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:45.457968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.457968 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:45.457968 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.457968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.457968 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:45.457968 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.457968 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:45.459431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.460228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:45.461357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.462157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.463010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:45.463010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.463010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:45.463010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:45.463010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:45.463282 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:45.463508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.464257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:23:45.464257 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.464257 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:45.464257 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.464257 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.464257 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:45.464257 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.464257 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:45.465588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.466487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:45.467271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.468140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:45.469091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.469091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:45.469091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:45.469091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:45.469091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:45.469091 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:45.469091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.469802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:45.469802 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.469802 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:45.469802 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.469802 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.469802 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:45.469802 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.469802 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.471407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:45.472311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:50.263442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:50.265039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:50.299414 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:50.299837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.300626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:23:50.300626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.300626 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:50.300626 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.300626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.300626 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:50.300626 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.300626 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.301575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.302543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:50.441904 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:50.442916 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:50.443263 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:50.443505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.444442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:50.444442 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.444442 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:50.444442 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.444442 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.444442 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:50.444442 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.444442 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.445783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.446394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:50.456352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:50.456876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:50.457404 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:50.457725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.458657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:50.458657 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.458657 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:50.458657 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.458657 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.458657 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:50.458657 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.458657 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:50.459998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.461078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:50.461700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.462690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.463653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:50.463653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.463653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:50.463653 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:50.463653 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:50.463653 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:50.463653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.464450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:23:50.464450 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.464450 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:50.464450 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.464450 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.464450 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:50.464450 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.464450 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:50.465604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.466259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:50.467354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.468072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:50.468795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.468795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:50.468795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:50.468795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:50.469133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:50.469454 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:50.469665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.470487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:50.470487 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.470487 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:50.470487 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.470487 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.470487 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:50.470487 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.470487 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.472169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:50.472914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:55.264079 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:55.265734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:55.295959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:55.296409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.297270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:23:55.297270 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.297270 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:55.297270 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.297270 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.297270 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:55.297270 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.297270 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.298154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.298964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:55.442593 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:55.443727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:55.443861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:55.444086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.444697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:23:55.444697 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.444697 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:55.444697 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.444697 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.444697 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:55.444697 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.444697 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.446398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.447439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:55.456847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:55.457469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:23:55.457891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:55.458119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.458750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:23:55.458750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.458750 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:55.458750 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.458750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.458750 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:55.458750 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.458750 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:55.460266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.461092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:23:55.461988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.462707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.463519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:55.463519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.463519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:55.463519 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:55.463919 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:55.464345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:55.464648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.465387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:23:55.465387 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.465387 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:55.465387 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.465387 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.465387 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:55.465387 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.465387 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:23:55.466651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.467170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:23:55.468171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.468635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:55.469174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.469174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:23:55.469174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:23:55.469174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:23:55.469174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:55.469364 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:23:55.469364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.469844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:23:55.469844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.469844 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:23:55.469844 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.469844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.469844 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:23:55.469844 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.469844 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.470963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:23:55.471529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:00.264744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:00.266882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:00.297913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:00.298198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.298857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:00.298857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.298857 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:00.298857 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.298857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.298857 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:00.298857 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.298857 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.300228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.301103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:00.443180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:00.444158 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:00.444487 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:00.444723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.445512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:00.445512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.445512 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:00.445512 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.445512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.445512 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:00.445512 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.445512 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.446693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.447180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:00.457272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:00.457763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:00.458321 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:00.458649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.459292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:24:00.459292 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.459292 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:00.459292 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.459292 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.459292 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:00.459292 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.459292 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:00.460636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:00.462006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.463011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:00.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:00.465047 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:00.465047 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:00.465423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.466261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:24:00.466261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.466261 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:00.466261 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.466261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.466261 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:00.466261 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.466261 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:00.467528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.468144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:00.469192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.469811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:00.470529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.470529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:00.470529 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:00.470529 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:00.470729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:00.471040 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:00.471279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.472141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:24:00.472141 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.472141 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:00.472141 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.472141 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.472141 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:00.472141 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.472141 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.473604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:00.474323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:05.265369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:05.266959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:05.297845 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:05.298070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.298567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:05.298567 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.298567 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:05.298567 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.298567 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.298567 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:05.298567 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.298567 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.299645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.300726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:05.443673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:05.445008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:05.445438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:05.445740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.446619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:05.446619 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.446619 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:05.446619 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.446619 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.446619 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:05.446619 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.446619 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.448438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.449470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:05.505443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:05.505443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:05.505443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:05.505708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:05.506132 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:05.506804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:05.507142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:05.507333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.507772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:24:05.507772 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.507772 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:05.507772 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.507772 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.507772 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:05.507772 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.507772 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:05.508447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:05.508938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:05.509637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.510371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.510957 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:05.511640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:05.512254 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:05.513254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.513918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.514610 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.515451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:05.515945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:10.265901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:10.267886 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:10.293629 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:10.293889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.294478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:10.294478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.294478 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:10.294478 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.294478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.294478 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:10.294478 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.294478 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.295345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.296414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:10.444125 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:10.445467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:10.445618 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:10.445690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.446095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:10.446095 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.446095 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:10.446095 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.446095 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.446095 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:10.446095 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.446095 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.446802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.447470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:10.505895 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:10.506144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:10.506762 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:10.507469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:10.507710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:10.507887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.508391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:24:10.508391 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.508391 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:10.508391 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.508391 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.508391 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:10.508391 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.508391 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:10.510325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:10.511437 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:10.512905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.513883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:10.515233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:10.516005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.516619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:10.517188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.517862 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:10.518310 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.518310 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:10.519317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:15.266939 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:15.268819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:15.305287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:15.305680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.306555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:15.306555 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.306555 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:15.306555 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.306555 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.306555 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:15.306555 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.306555 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.307739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.308852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:15.445349 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:15.446539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:15.446832 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:15.447121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.447910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:15.447910 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.447910 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:15.447910 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.447910 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.447910 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:15.447910 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.447910 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.449571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.450559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:15.506770 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:15.507326 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:15.507538 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:15.508358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:15.508568 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:15.508842 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.509749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:15.510170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.510687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:24:15.510687 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.510687 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:15.510687 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.510687 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.510687 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:15.510687 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.510687 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:15.512740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:15.513652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:15.515585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.517384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:15.518218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.519058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:15.520115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:15.520491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Traceback (most recent call last):
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.521562 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-8eba8d02-065d-4779-b6ca-be9aaee3e5f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.522777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:15.523359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:20.267672 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:20.269351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:20.304258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:20.304463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.305106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:20.305106 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.305106 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:20.305106 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.305106 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.305106 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:20.305106 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.305106 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.306322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.307289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:20.445940 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:20.447015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:20.447327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:20.447547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.448530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:20.448530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.448530 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:20.448530 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.448530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.448530 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:20.448530 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.448530 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.450349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.451242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:20.507403 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:20.508440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:20.509044 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:20.509793 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:20.510283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:20.510618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.511462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:24:20.511462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.511462 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:20.511462 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.511462 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.511462 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:20.511462 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.511462 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:20.512994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:20.513821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:20.515336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.516177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:20.517099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:20.518502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:20.519325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:20.520553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.521116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.521626 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.522990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:20.523529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:25.268380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:25.270103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:25.309563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:25.310522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.310788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:25.310788 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.310788 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:25.310788 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.310788 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.310788 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:25.310788 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.310788 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.312470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.313482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:25.446667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:25.447864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:25.448400 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:25.448660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.449607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:25.449607 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.449607 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:25.449607 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.449607 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.449607 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:25.449607 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.449607 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.451371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:25.508261 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:25.508850 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:25.509668 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:25.510218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:25.510694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.511304 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:25.512171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:25.512171 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.512171 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:25.512171 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.512171 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.512171 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:25.512171 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.512171 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:25.513579 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:25.514135 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:25.515691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.516168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.516662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:25.516662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.516662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:25.516662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.516662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:25.516937 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:25.517730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:24:25.517730 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.517730 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:25.517730 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.517730 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.517730 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:25.517730 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.517730 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:25.519350 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:25.519752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:25.521500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.521967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:25.522468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.522468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:25.522468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:25.522468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.522587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:25.522587 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.522587 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:25.522587 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.522587 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.522587 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:25.522587 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.522587 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.523799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:25.524332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:30.268918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:30.270768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:30.273866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:24:30.307336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:30.307692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.308531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:30.308531 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.308531 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:30.308531 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.308531 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.308531 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:30.308531 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.308531 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.309521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.310453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:30.447371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:30.448744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:30.448744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:30.449087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:30.449487 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.449487 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:30.449487 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.449487 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.449487 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:30.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.451225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.452445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:30.508944 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:30.509453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:30.509924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:30.510261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.510742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:30.511416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:24:30.511416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.511416 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:30.511416 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.511416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.511416 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:30.511416 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.511416 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:30.511955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:30.513301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:30.514258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:30.514883 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.515880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.516964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.518231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.518822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:30.519502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:30.520167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.520530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:30.521210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:30.521903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:30.521903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:30.521903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:30.521903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:35.269706 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:35.272697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:35.306258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:35.306681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.307478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:35.307478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.307478 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:35.307478 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.307478 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.307478 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:35.307478 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.307478 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.308318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.309231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:35.448031 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:35.449331 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:35.449612 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:35.449843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.450693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:35.450693 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.450693 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:35.450693 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.450693 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.450693 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:35.450693 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.450693 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.452519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.453348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:35.509511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:35.510226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:35.510901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:35.511291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.512150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:35.512370 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:35.512720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:24:35.512720 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.512720 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:35.512720 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.512720 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.512720 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:35.512720 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.512720 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:35.514293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:35.515214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:35.516407 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.517319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.518239 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:35.518934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:35.519841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:35.520951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.521663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:35.522315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.522315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:35.522315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:35.522315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:35.522315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:35.522315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.523019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:24:35.523019 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.523019 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:35.523019 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.523019 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.523019 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:35.523019 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.523019 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.524079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:35.524740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:40.270463 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:40.272711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:40.302047 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:40.302478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.303383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:40.303383 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.303383 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:40.303383 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.303383 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.303383 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:40.303383 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.303383 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.304580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.305489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:40.448751 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:40.449764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:40.450139 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:40.450380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.451111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:40.451111 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.451111 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:40.451111 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.451111 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.451111 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:40.451111 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.451111 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.452226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.453143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:40.510253 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:40.511143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:40.511675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:40.511901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.512635 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:40.514069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:24:40.514069 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.514069 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:40.514069 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.514069 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.514069 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:40.514069 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.514069 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:40.515441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:40.516514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:40.517792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.518799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.519513 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:40.520604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:40.521047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:40.521047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.521047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:40.521047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.521047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.521047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:40.521047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.521047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:40.522101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:40.523060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.523060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:40.523060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:40.523252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:40.523943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:40.524303 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:40.524520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.525217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:24:40.525217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.525217 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:40.525217 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.525217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.525217 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:40.525217 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.525217 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.526492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:40.526930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:45.271650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:45.273318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:45.303592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:45.303879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.304541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:45.304541 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.304541 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:45.304541 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.304541 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.304541 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:45.304541 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.304541 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.305466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.306502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:45.449339 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:45.450703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:45.451134 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:45.451508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.452495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:45.452495 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.452495 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:45.452495 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.452495 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.452495 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:45.452495 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.452495 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.454386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.454956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:45.511182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:45.511736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:45.512435 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:45.512783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.513690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:24:45.513690 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.513690 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:45.513690 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.513690 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.513690 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:45.513690 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.513690 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:45.515044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.515927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:45.516427 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:45.517650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.518139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:45.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:45.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:45.518648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:45.518944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.519749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:45.519749 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.519749 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:45.519749 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.519749 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.519749 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:45.519749 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.519749 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:45.520602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:45.521066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:45.521066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.521066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:45.521066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.521066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.521066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:45.521066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.521066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:45.522110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.522557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:45.523026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.523026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:45.523175 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:45.523487 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:45.523805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:45.524094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:45.524255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.524718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:45.524718 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.524718 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:45.524718 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.524718 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.524718 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:45.524718 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.524718 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.526286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:45.526813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:50.272313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:50.273810 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:50.311332 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:50.311721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.312663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:50.312663 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.312663 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:50.312663 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.312663 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.312663 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:50.312663 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.312663 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.313906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.314927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:50.450071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:50.451385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:50.451736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:50.451983 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.453105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:50.453105 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.453105 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:50.453105 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.453105 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.453105 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:50.453105 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.453105 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.454863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.455454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:50.511702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:50.512271 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:50.512932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:50.513202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.514018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:24:50.514018 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.514018 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:50.514018 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.514018 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.514018 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:50.514018 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.514018 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:50.515306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.516094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:50.516904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.517504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.518112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:50.518112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.518112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:50.518112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:50.518112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:50.518292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:50.518535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.519171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:24:50.519171 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.519171 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:50.519171 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.519171 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.519171 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:50.519171 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.519171 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:50.520411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.520864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:50.522088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.522656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:50.523245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.523245 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:50.523552 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:50.523819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:50.524391 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:50.524838 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:50.525143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.525987 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:50.525987 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.525987 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:50.525987 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.525987 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.525987 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:50.525987 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.525987 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.527153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:50.527720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:55.273008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:55.274404 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:55.311599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:55.311776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.312193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:24:55.312193 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.312193 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:55.312193 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.312193 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.312193 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:55.312193 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.312193 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.313032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.313792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:55.450794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:55.452221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:55.452586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:55.452919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.453733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:55.453733 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.453733 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:55.453733 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.453733 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.453733 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:55.453733 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.453733 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.455519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.456656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:55.512473 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:55.512960 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:24:55.513638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:55.513999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.514789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:24:55.514789 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.514789 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:55.514789 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.514789 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.514789 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:55.514789 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.514789 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:55.515766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.516697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:24:55.517479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.519099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.520315 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:24:55.521191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.521891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:24:55.522384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.523035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:55.523744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.523744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:24:55.523881 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:24:55.524232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:24:55.524533 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:55.524750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:24:55.524898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.525405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:24:55.525405 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.525405 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:24:55.525405 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.525405 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.525405 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:24:55.525405 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.525405 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.527049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:24:55.527718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:00.273784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:00.275218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:00.312785 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:00.313100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.313812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:00.313812 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.313812 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:00.313812 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.313812 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.313812 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:00.313812 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.313812 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.315091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.315950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:00.451773 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:00.453066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:00.453386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:00.453604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.454320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:00.454320 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.454320 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:00.454320 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.454320 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.454320 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:00.454320 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.454320 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.455688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.456473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:00.513538 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:00.514327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:00.514906 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:00.515290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.516185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:25:00.516185 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.516185 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:00.516185 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.516185 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.516185 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:00.516185 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.516185 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:00.517515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.518329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:00.519492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.520232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:00.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:00.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:00.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:00.521345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:00.521835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.522586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:25:00.522586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.522586 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:00.522586 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.522586 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.522586 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:00.522586 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.522586 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:00.523408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.523836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:00.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.525458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:00.526038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.526038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:00.526038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:00.526038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:00.526038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:00.526328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:00.526454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.526902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:25:00.526902 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.526902 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:00.526902 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.526902 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.526902 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:00.526902 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.526902 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.528010 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:00.528577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:05.274424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:05.275284 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:05.314123 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:05.314583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.315427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:05.315427 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.315427 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:05.315427 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.315427 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.315427 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:05.315427 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.315427 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.316606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.317432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:05.452457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:05.453819 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:05.454324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:05.454617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.455665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:05.455665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.455665 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:05.455665 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.455665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.455665 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:05.455665 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.455665 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.457481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.458267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:05.514446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:05.515612 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:05.516522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:05.516832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.517703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:05.517703 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.517703 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:05.517703 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.517703 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.517703 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:05.517703 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.517703 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:05.519314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:05.520435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:05.520435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.520435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:05.520435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.520435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.520435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:05.520435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.520435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:05.521038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.522029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.523238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:05.523238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.523238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:05.523238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:05.523238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:05.523563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:05.523929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.524768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:05.524768 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.524768 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:05.524768 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.524768 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.524768 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:05.524768 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.524768 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:05.527904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:05.528376 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:05.530333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.530857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:05.531446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.531446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:05.531446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:05.531446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:05.531588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:05.531758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.533057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Traceback (most recent call last):
Mar 26 01:25:05.533057 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.533057 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:05.533057 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.533057 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.533057 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:05.533057 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.533057 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-f1353043-1353-47a4-b874-ccdc5ee82da6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.534567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:05.535176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:10.275035 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:10.276749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:10.311495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:10.311905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.313024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:10.313024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.313024 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:10.313024 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.313024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.313024 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:10.313024 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.313024 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.314401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.314912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:10.453179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:10.454528 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:10.454851 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:10.455117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.455907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:10.455907 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.455907 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:10.455907 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.455907 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.455907 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:10.455907 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.455907 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.457742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.458950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:10.515022 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:10.515584 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:10.516338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:10.516664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.517449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:10.517449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.517449 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:10.517449 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.517449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.517449 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:10.517449 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.517449 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:10.518856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:10.520027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:10.520027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.520027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:10.520027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.520027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.520027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:10.520027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.520027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:10.520658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.521687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.522877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:10.522877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.522877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:10.522877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:10.523201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:10.523554 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:10.523961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.524902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:10.524902 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.524902 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:10.524902 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.524902 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.524902 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:10.524902 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.524902 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:10.526603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:10.527159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:10.527159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.527159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:10.527159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.527159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.527159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:10.527159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.527159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:10.528701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.529359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:10.530064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.530064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:10.530064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:10.530064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:10.530365 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:10.530831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:10.531108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.531850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:25:10.531850 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.531850 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:10.531850 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.531850 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.531850 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:10.531850 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.531850 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.533492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:10.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:15.276045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:15.277920 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:15.311283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:15.311554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.311927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:15.311927 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.311927 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:15.311927 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.311927 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.311927 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:15.311927 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.311927 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.313037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.313886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:15.454223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:15.455723 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:15.456019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:15.456321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.457369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:15.457369 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.457369 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:15.457369 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.457369 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.457369 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:15.457369 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.457369 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.459466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.460286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:15.516259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:15.516972 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:15.517657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:15.518014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.519188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:15.519188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.519188 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:15.519188 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.519188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.519188 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:15.519188 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.519188 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:15.520473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:15.521263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:15.521263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.521263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:15.521263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.521263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.521263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:15.521263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.521263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:15.522368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.523114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.523903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:15.523903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.523903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:15.523903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:15.523903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:15.524460 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:15.524744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:15.525515 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.525515 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:15.525515 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.525515 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.525515 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:15.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:15.527119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:15.527893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:15.527893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.527893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:15.527893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.527893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.527893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:15.527893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.527893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:15.529256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.529748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:15.530320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.530320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:15.531003 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:15.531378 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:15.531918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:15.532867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:15.533031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.534041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:25:15.534041 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.534041 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:15.534041 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.534041 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.534041 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:15.534041 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.534041 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.535213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:15.535827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:20.276996 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:20.279536 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:20.315443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:20.315731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.316628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:20.316628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.316628 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:20.316628 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.316628 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.316628 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:20.316628 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.316628 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.317850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.318396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:20.455018 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:20.456097 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:20.456443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:20.456750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.457643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:20.457643 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.457643 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:20.457643 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.457643 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.457643 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:20.457643 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.457643 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.458770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.459542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:20.516946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:20.517512 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:20.518230 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:20.518901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.519428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:20.519428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.519428 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:20.519428 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.519428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.519428 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:20.519428 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.519428 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:20.520822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:20.521676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:20.521676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.521676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:20.521676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.521676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.521676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:20.521676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.521676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:20.522545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.523279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.524072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:20.524072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.524072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:20.524072 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:20.524072 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:20.524348 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:20.524693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.525487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:20.525487 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.525487 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:20.525487 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.525487 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.525487 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:20.525487 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.525487 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:20.526281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:20.527233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:20.527233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.527233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:20.527233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.527233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.527233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:20.527233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.527233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:20.527623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.528362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:20.529244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.529244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:20.531300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:20.531579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:20.531853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:20.532053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:20.532169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:20.532845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:20.534963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:20.535561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:20.535561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:20.535561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:20.535561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:20.535561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:20.535561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:25.277801 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:25.279195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:25.315514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:25.315761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.316357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:25.316357 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.316357 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:25.316357 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.316357 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.316357 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:25.316357 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.316357 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.318433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.319288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:25.455713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:25.456885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:25.457235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:25.457465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.458359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:25.458359 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.458359 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:25.458359 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.458359 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.458359 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:25.458359 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.458359 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.460146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.460920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:25.517598 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:25.517809 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:25.518499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:25.518790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.519567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:25.519567 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.519567 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:25.519567 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.519567 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.519567 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:25.519567 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.519567 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:25.520464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:25.521312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:25.521312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.521312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:25.521312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.521312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.521312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:25.521312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.521312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:25.521686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.522424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.523266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:25.523266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.523266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:25.523780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:25.523954 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:25.524500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:25.524967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.525855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:25.525855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.525855 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:25.525855 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.525855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.525855 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:25.525855 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.525855 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:25.526559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:25.527040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:25.527040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.527040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:25.527040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.527040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.527040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:25.527040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.527040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:25.527554 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.528133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:25.528663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.528663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:25.531731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:25.532435 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:25.533056 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:25.533056 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:25.533294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.533772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:25:25.533772 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.533772 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:25.533772 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.533772 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.533772 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:25.533772 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.533772 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.534906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:25.535474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:30.278520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:30.280659 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:30.283524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:25:30.315673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:30.316000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.316792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:30.316792 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.316792 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:30.316792 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.316792 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.316792 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:30.316792 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.316792 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.318012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.319140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:30.456237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:30.457421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:30.457729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:30.457954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.458841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:30.458841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.458841 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:30.458841 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.458841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.458841 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:30.458841 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.458841 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.460889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.461267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:30.518041 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:30.518371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:30.518867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:30.519704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.520309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:30.520309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.520309 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:30.520309 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.520309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.520309 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:30.520309 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.520309 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:30.521029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:30.521451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:30.521451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.521451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:30.521451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.521451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.521451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:30.521451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.521451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:30.521734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.522107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.522461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:30.522461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.522461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:30.524063 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:30.524345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:30.524663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:30.525019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.525399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:30.525399 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.525399 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:30.525399 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.525399 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.525399 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:30.525399 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.525399 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:30.526139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:30.526904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:30.526904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.526904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:30.526904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.526904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.526904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:30.526904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.526904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:30.527502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.528163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:30.528748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.528748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:30.532299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:30.532837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:30.533260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:30.533583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:30.533796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.534540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:25:30.534540 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.534540 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:30.534540 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.534540 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.534540 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:30.534540 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.534540 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.536235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:30.537069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:35.279366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:35.280849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:35.317531 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:35.317694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.318083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:35.318083 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.318083 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:35.318083 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.318083 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.318083 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:35.318083 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.318083 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.318828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.319609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:35.456958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:35.457946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:35.458290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:35.458518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.459484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:35.459484 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.459484 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:35.459484 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.459484 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.459484 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:35.459484 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.459484 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.461144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.462021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:35.518699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:35.519412 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:35.519993 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:35.520329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.521202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:35.521202 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.521202 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:35.521202 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.521202 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.521202 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:35.521202 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.521202 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:35.523163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:35.524131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:35.524131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.524131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:35.524131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.524131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.524131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:35.524131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.524131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:35.525277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.526044 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.526708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:35.526708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.526708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:35.526708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:35.526708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:35.526708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:35.527160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.527829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:35.527829 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.527829 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:35.527829 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.527829 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.527829 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:35.527829 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.527829 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:35.529171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:35.529947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:35.529947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.529947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:35.529947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.529947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.529947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:35.529947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.529947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:35.531112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.531818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:35.532682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.532682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:35.532787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:35.533238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:35.533629 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:35.534059 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:35.534200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.534956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:25:35.534956 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.534956 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:35.534956 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.534956 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.534956 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:35.534956 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.534956 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.536583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:35.537447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:40.280120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:40.281622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:40.318935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:40.319158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.319496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:40.319496 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.319496 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:40.319496 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.319496 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.319496 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:40.319496 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.319496 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.321658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:40.457923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:40.459462 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:40.459890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:40.460275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.461309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:40.461309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.461309 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:40.461309 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.461309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.461309 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:40.461309 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.461309 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.463278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.463970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:40.519277 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:40.519591 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:40.520173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:40.520463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.521338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:40.521338 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.521338 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:40.521338 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.521338 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.521338 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:40.521338 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.521338 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:40.522826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:40.524145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:40.524145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.524145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:40.524145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.524145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.524145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:40.524145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.524145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:40.525069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.525891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.526463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:40.526463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.526463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:40.526463 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:40.526463 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:40.526852 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:40.527241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.527923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:40.527923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.527923 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:40.527923 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.527923 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.527923 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:40.527923 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.527923 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:40.529226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:40.529688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:40.529688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.529688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:40.529688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.529688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.529688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:40.529688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.529688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:40.530671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.531170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:40.531667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.531667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:40.533064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:40.533686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:40.534053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:40.534299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:40.534465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.535094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:25:40.535094 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.535094 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:40.535094 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.535094 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.535094 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:40.535094 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.535094 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.536710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:40.537222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:45.280777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:45.282771 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:45.313798 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:45.314251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.315160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:45.315160 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.315160 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:45.315160 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.315160 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.315160 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:45.315160 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.315160 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.316058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.316860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:45.458634 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:45.459955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:45.460329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:45.460561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.461463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:45.461463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.461463 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:45.461463 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.461463 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.461463 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:45.461463 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.461463 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.463270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.463785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:45.519895 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:45.520478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:45.521079 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:45.521358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.522179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:45.522179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.522179 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:45.522179 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.522179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.522179 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:45.522179 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.522179 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:45.523428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:45.524002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:45.524002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.524002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:45.524002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.524002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.524002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:45.524002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.524002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:45.525008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.525510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.526285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:45.526285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.526285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:45.526501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:45.526918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:45.527396 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:45.527729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.528379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:45.528379 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.528379 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:45.528379 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.528379 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.528379 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:45.528379 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.528379 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:45.528991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:45.529472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:45.529472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.529472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:45.529472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.529472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.529472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:45.529472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.529472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:45.529697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.530200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:45.530739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.530739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:45.533550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:45.533867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:45.534286 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:45.534632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:45.534832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.536065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:25:45.536065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.536065 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:45.536065 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.536065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.536065 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:45.536065 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.536065 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.537509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:45.539680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:50.281459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:50.282079 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:50.314414 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:50.314781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.315717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:50.315717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.315717 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:50.315717 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.315717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.315717 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:50.315717 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.315717 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.317045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.317899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:50.459401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:50.460796 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:50.461182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:50.461427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.462343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:50.462343 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.462343 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:50.462343 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.462343 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.462343 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:50.462343 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.462343 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.464275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.465099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:50.520598 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:50.521113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:50.521664 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:50.521948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.522708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:50.522708 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.522708 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:50.522708 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.522708 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.522708 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:50.522708 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.522708 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:50.523937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:50.524928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:50.524928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.524928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:50.524928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.524928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.524928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:50.524928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.524928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:50.525515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.526232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.527034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:50.527034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.527034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:50.527034 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:50.527678 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:50.528200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:50.528545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.529566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:50.529566 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.529566 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:50.529566 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.529566 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.529566 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:50.529566 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.529566 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:50.530230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:50.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:50.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:50.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:50.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:50.531604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.532356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:50.533182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.533182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:50.533844 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:50.534399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:50.534801 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:50.535165 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:50.535379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.536092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:25:50.536092 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.536092 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:50.536092 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.536092 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.536092 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:50.536092 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.536092 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.537682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:50.538187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:55.282186 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:55.284041 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:55.325578 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:55.325913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.326814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:25:55.326814 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.326814 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:55.326814 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.326814 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.326814 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:55.326814 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.326814 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.328399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.330119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:55.459948 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:55.461606 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:55.461675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:55.461912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.463016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:55.463016 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.463016 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:55.463016 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.463016 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.463016 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:55.463016 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.463016 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.464646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.465450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:55.521226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:55.521677 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:25:55.522161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:55.522370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.522871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:55.522871 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.522871 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:55.522871 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.522871 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.522871 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:55.522871 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.522871 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:55.523507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.524422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:55.524990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.525624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:55.527569 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:55.528053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:55.528432 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:55.529988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.530968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:25:55.530968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.530968 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:55.530968 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.530968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.530968 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:55.530968 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.530968 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:25:55.532757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:55.533235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:25:55.533235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.533235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:25:55.533235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.533235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.533235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:25:55.533235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.533235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:25:55.534377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.535140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:55.535648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.535648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:25:55.535648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:25:55.535648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:25:55.535648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:55.535857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:25:55.536026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.537231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:25:55.537231 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.537231 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:25:55.537231 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.537231 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.537231 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:25:55.537231 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.537231 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.539234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:25:55.539943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:00.282781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:00.283846 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:00.319737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:00.320131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.321055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:00.321055 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.321055 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:00.321055 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.321055 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.321055 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:00.321055 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.321055 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.322238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.323105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:00.460739 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:00.461775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:00.462166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:00.462433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.463261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:00.463261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.463261 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:00.463261 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.463261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.463261 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:00.463261 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.463261 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.464441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.465262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:00.521840 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:00.522247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:00.522895 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:00.523236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.524004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:00.524004 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.524004 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:00.524004 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.524004 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.524004 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:00.524004 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.524004 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:00.525357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:00.526174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:00.526174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.526174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:00.526174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.526174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.526174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:00.526174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.526174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:00.527409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.528199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.529002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:00.529002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.529002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:00.529002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:00.529002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:00.529002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:00.529002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.529351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:00.529351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.529351 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:00.529351 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.529351 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.529351 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:00.529351 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.529351 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:00.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:00.531406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:00.531406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.531406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:00.531406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.531406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.531406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:00.531406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.531406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:00.532062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.532893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:00.534335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.534335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:00.535312 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:00.535992 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:00.536410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:00.536634 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:00.536932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.537413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:00.537413 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.537413 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:00.537413 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.537413 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.537413 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:00.537413 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.537413 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.539423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:00.540792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:05.283607 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:05.285242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:05.320305 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:05.320540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.320993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:05.320993 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.320993 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:05.320993 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.320993 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.320993 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:05.320993 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.320993 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.322075 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.323067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:05.461342 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:05.462325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:05.462500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:05.462637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.463235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:05.463235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.463235 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:05.463235 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.463235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.463235 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:05.463235 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.463235 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.465007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.465775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:05.522312 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:05.522592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:05.523090 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:05.523368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.524090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:05.524090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.524090 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:05.524090 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.524090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.524090 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:05.524090 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.524090 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:05.525465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:05.526400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:05.526400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.526400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:05.526400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.526400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.526400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:05.526400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.526400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:05.526878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.527770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.528707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:05.528707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.528707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:05.528707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:05.528707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:05.529066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:05.529286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.529717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:05.529717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.529717 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:05.529717 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.529717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.529717 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:05.529717 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.529717 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:05.531119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:05.531545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:05.531545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.531545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:05.531545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.531545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.531545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:05.531545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.531545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:05.532838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.533351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:05.533873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.533873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:05.535761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:05.536114 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:05.536651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:05.536651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:05.536753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.537267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:05.537267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.537267 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:05.537267 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.537267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.537267 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:05.537267 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.537267 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.538232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:05.538764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:10.284219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:10.285285 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:10.321580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:10.322119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.323127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:10.323127 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.323127 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:10.323127 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.323127 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.323127 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:10.323127 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.323127 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.324339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.324887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:10.462007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:10.463397 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:10.463728 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:10.463965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.464939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:10.464939 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.464939 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:10.464939 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.464939 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.464939 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:10.464939 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.464939 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.466910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.467707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:10.522854 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:10.523278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:10.523998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:10.524391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.525307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:10.525307 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.525307 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:10.525307 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.525307 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.525307 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:10.525307 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.525307 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:10.526776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:10.527611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:10.527611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.527611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:10.527611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.527611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.527611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:10.527611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.527611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:10.528818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.529635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.530468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:10.530468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.530468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:10.530468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:10.530468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:10.530721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:10.530996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.531714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:10.531714 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.531714 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:10.531714 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.531714 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.531714 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:10.531714 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.531714 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:10.532441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:10.533193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:10.533193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.533193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:10.533193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.533193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.533193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:10.533193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.533193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:10.533503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.534024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:10.534607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.534607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:10.536188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:10.536758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:10.537143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:10.537421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:10.537596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.538320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:10.538320 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.538320 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:10.538320 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.538320 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.538320 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:10.538320 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.538320 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:10.540739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:15.284883 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:15.286943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:15.320338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:15.320690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.321149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:15.321149 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.321149 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:15.321149 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.321149 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.321149 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:15.321149 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.321149 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.322213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.322899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:15.462739 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:15.463841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:15.464421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:15.464535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.465449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:15.465449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.465449 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:15.465449 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.465449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.465449 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:15.465449 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.465449 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.466695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.467633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:15.523309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:15.524010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:15.524682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:15.525003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.526855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:15.526855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.526855 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:15.526855 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.526855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.526855 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:15.526855 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.526855 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:15.527776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.528736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:15.530283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:15.531414 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.531849 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:15.532623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:15.533232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:15.533232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.533232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:15.533232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.533232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.533232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:15.533232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.533232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:15.534375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.535037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:15.535684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.535684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:15.536480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:15.537094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:15.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:15.537477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:15.537852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.538323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:15.538323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.538323 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:15.538323 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.538323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.538323 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:15.538323 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.538323 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.539755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:15.540625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:20.285475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:20.287141 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:20.320339 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:20.320727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.321545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:20.321545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.321545 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:20.321545 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.321545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.321545 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:20.321545 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.321545 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.322382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.323342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:20.463386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:20.464376 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:20.464550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:20.464655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.465405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:20.465405 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.465405 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:20.465405 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.465405 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.465405 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:20.465405 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.465405 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.466761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.467646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:20.523893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:20.524378 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:20.525157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:20.525464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.526218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:20.526218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.526218 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:20.526218 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.526218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.526218 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:20.526218 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.526218 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:20.527058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.528062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:20.528850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:20.529801 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:20.530398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:20.530543 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:20.530870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.531522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:20.531522 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.531522 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:20.531522 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.531522 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.531522 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:20.531522 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.531522 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:20.532691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:20.533121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:20.533121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.533121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:20.533121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.533121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.533121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:20.533121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.533121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:20.533779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.534254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:20.534748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.534748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:20.536879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:20.537287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:20.537618 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:20.537869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:20.538018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.538480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:20.538480 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.538480 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:20.538480 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.538480 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.538480 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:20.538480 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.538480 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.539494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:20.540034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:25.286291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:25.288267 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:25.323703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:25.324129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.324971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:25.324971 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.324971 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:25.324971 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.324971 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.324971 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:25.324971 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.324971 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.326306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.327465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:25.463923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:25.465362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:25.465759 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:25.465937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.466574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:25.466574 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.466574 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:25.466574 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.466574 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.466574 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:25.466574 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.466574 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.468469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.468803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:25.524545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:25.525367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:25.526109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:25.526523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.527576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:25.527576 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.527576 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:25.527576 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.527576 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.527576 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:25.527576 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.527576 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:25.528900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:25.529749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:25.529749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.529749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:25.529749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.529749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.529749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:25.529749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.529749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:25.530941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.532030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.532856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:25.532856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.532856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:25.532856 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:25.532856 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:25.532856 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:25.532856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.533431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:25.533431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.533431 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:25.533431 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.533431 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.533431 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:25.533431 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.533431 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:25.534395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:25.535283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:25.535283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.535283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:25.535283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.535283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.535283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:25.535283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.535283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:25.535709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.536728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:25.537627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.537627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:25.537627 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:25.537627 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:25.538000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:25.538216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:25.538422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.538934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:25.538934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.538934 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:25.538934 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.538934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.538934 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:25.538934 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.538934 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.540164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:25.540967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:30.286771 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:30.287832 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:30.293148 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=241 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:26:30.317510 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:30.317960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.318890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:30.318890 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.318890 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:30.318890 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.318890 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.318890 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:30.318890 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.318890 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.320543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.321497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:30.464700 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:30.465774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:30.466068 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:30.466292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.466994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:30.466994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.466994 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:30.466994 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.466994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.466994 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:30.466994 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.466994 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.467790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.468576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:30.525251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:30.525824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:30.526451 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:30.526743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.527540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:30.527540 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.527540 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:30.527540 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.527540 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.527540 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:30.527540 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.527540 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:30.528943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:30.529755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:30.529755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.529755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:30.529755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.529755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.529755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:30.529755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.529755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:30.530845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.531413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.531956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:30.531956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.531956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:30.531956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:30.532313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:30.532844 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:30.533370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.534120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:30.534120 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.534120 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:30.534120 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.534120 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.534120 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:30.534120 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.534120 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:30.535427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:30.535933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:30.535933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.535933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:30.535933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.535933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.535933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:30.535933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.535933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:30.537028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.537495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:30.538013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.538013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:30.538013 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:30.538638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:30.539165 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:30.539653 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:30.539947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.540798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:26:30.540798 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.540798 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:30.540798 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.540798 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.540798 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:30.540798 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.540798 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.543008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:30.543592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:35.287517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:35.288967 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:35.325241 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:35.325517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.326376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:35.326376 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.326376 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:35.326376 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.326376 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.326376 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:35.326376 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.326376 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.328270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:35.465436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:35.466582 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:35.466931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:35.467219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.467994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:35.467994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.467994 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:35.467994 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.467994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.467994 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:35.467994 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.467994 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.469856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.470656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:35.525938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:35.526395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:35.527037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:35.528028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.528232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:35.528232 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.528232 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:35.528232 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.528232 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.528232 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:35.528232 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.528232 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:35.529561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:35.530512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:35.530512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.530512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:35.530512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.530512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.530512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:35.530512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.530512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:35.531336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.532165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.533738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:35.533738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.533738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:35.533738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:35.533738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:35.533738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:35.534177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.534568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:35.534568 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.534568 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:35.534568 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.534568 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.534568 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:35.534568 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.534568 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:35.535436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:35.536136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:35.536136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.536136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:35.536136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.536136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.536136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:35.536136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.536136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:35.536903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:35.538020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.538020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:35.538020 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:35.538306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:35.538647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:35.538932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:35.539162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.539696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:35.539696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.539696 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:35.539696 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.539696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.539696 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:35.539696 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.539696 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.540786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:35.541425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:40.287956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:40.289661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:40.318288 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:40.318511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.319194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:40.319194 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.319194 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:40.319194 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.319194 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.319194 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:40.319194 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.319194 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.320045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.320837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:40.466208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:40.467343 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:40.467544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:40.467685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.468248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:40.468248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.468248 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:40.468248 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.468248 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.468248 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:40.468248 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.468248 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.469330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.470082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:40.526643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:40.526879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:40.527495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:40.527802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.528535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:40.528535 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.528535 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:40.528535 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.528535 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.528535 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:40.528535 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.528535 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:40.529330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:40.530259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:40.530259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.530259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:40.530259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.530259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.530259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:40.530259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.530259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:40.530684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.531541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.532357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:40.532357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.532357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:40.532587 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:40.533201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:40.533749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:40.534114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.534705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:40.534705 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.534705 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:40.534705 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.534705 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.534705 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:40.534705 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.534705 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:40.535719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:40.536172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:40.536172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.536172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:40.536172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.536172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.536172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:40.536172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.536172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:40.536735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.537209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:40.537713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.537713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:40.538320 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:40.538523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:40.538788 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:40.539062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:40.539277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.539743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:40.539743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.539743 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:40.539743 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.539743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.539743 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:40.539743 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.539743 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.540685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:40.541201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:45.288703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:45.290350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:45.325777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:45.326083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.326775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:45.326775 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.326775 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:45.326775 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.326775 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.326775 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:45.326775 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.326775 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.327767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.328841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:45.467036 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:45.468360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:45.468640 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:45.469104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.469724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:45.469724 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.469724 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:45.469724 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.469724 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.469724 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:45.469724 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.469724 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.471752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.472609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:45.527142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:45.527748 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:45.527999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:45.528291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.529034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:45.529034 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.529034 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:45.529034 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.529034 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.529034 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:45.529034 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.529034 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:45.530358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:45.531270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:45.531270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.531270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:45.531270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.531270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.531270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:45.531270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.531270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:45.531917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.532689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.533243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:45.533243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.533243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:45.533243 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:45.533724 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:45.534345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:45.534703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.535458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:45.535458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.535458 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:45.535458 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.535458 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.535458 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:45.535458 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.535458 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:45.536618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:45.537067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:45.537067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.537067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:45.537067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.537067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.537067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:45.537067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.537067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:45.538201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.538680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:45.539257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.539257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:45.539257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:45.539257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:45.539504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:45.539857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:45.540134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.540862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:45.540862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.540862 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:45.540862 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.540862 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.540862 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:45.540862 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.540862 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.542009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:45.542537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:50.289295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:50.291090 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:50.321455 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:50.321837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.322683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:50.322683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.322683 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:50.322683 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.322683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.322683 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:50.322683 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.322683 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.323595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.324195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:50.467883 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:50.469122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:50.469399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:50.469617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.470344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:50.470344 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.470344 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:50.470344 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.470344 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.470344 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:50.470344 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.470344 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.471908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.472405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:50.527661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:50.527937 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:50.528473 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:50.528747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.529410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:50.529410 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.529410 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:50.529410 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.529410 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.529410 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:50.529410 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.529410 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:50.530521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:50.531544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:50.531544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.531544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:50.531544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.531544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.531544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:50.531544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.531544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:50.531948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.532717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.533716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:50.533716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.533716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:50.533716 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:50.533917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:50.534466 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:50.534942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.535687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:50.535687 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.535687 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:50.535687 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.535687 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.535687 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:50.535687 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.535687 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:50.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:50.537522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:50.537522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.537522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:50.537522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.537522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.537522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:50.537522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.537522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:50.538412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.539098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:50.539624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.539624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:50.539624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:50.539624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:50.539854 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:50.540283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:50.540510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.541242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:50.541242 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.541242 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:50.541242 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.541242 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.541242 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:50.541242 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.541242 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.542258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:50.542808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:55.290005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:55.291638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:55.322613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:55.323004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.323918 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:26:55.323918 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.323918 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:55.323918 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.323918 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.323918 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:55.323918 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.323918 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.325272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.326649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:55.468572 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:55.469779 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:55.470088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:55.470598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.471066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:55.471066 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.471066 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:55.471066 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.471066 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.471066 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:55.471066 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.471066 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.472572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.473534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:55.528238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:55.528813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:26:55.529521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:55.529888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.530705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:55.530705 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.530705 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:55.530705 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.530705 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.530705 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:55.530705 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.530705 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:55.531953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:55.532835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:55.532835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.532835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:55.532835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.532835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.532835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:55.532835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.532835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:26:55.533488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.534304 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.535121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:55.535121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.535121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:55.535121 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:55.535121 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:55.535121 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:55.535455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.536070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:26:55.536070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.536070 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:55.536070 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.536070 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.536070 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:55.536070 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.536070 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:26:55.537364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:55.538130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:26:55.538130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.538130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:26:55.538130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.538130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.538130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:26:55.538130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.538130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:26:55.538721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.539226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:55.539735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.539735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:26:55.539735 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:26:55.539735 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:26:55.540153 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:55.540423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:26:55.540631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.541400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:26:55.541400 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.541400 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:26:55.541400 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.541400 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.541400 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:26:55.541400 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.541400 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.542398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:26:55.542879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:00.290654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:00.292291 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:00.327329 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:00.327567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.328355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:00.328355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.328355 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:00.328355 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.328355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.328355 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:00.328355 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.328355 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.329268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.330214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:00.469325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:00.470621 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:00.470889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:00.471191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.471897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:00.471897 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.471897 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:00.471897 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.471897 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.471897 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:00.471897 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.471897 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.473545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.474448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:00.528821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:00.529105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:00.529786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:00.530155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:00.530900 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.530900 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:00.530900 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.530900 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.530900 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:00.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.530900 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:00.532194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:00.532962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:00.532962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.532962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:00.532962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.532962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.532962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:00.532962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.532962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:00.533678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.534433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.535273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:00.535273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.535273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:00.535273 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:00.535273 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:00.535521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:00.535696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.536417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:00.536417 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.536417 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:00.536417 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.536417 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.536417 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:00.536417 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.536417 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:00.537654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:00.538373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:00.538373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.538373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:00.538373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.538373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.538373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:00.538373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.538373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:00.539345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.539820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:00.540374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.540374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:00.540374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:00.540374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:00.540588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:00.541011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:00.541296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.542068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:00.542068 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.542068 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:00.542068 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.542068 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.542068 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:00.542068 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.542068 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.543650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:00.544161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:05.291386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:05.292901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:05.327224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:05.327552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.328188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:05.328188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.328188 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:05.328188 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.328188 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.328188 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:05.328188 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.328188 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.329056 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.329604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:05.470013 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:05.472035 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:05.472035 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:05.472035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.472330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:05.472330 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.472330 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:05.472330 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.472330 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.472330 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:05.472330 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.472330 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.474034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.474523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:05.529350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:05.529761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:05.530478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:05.530824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.532106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:05.532106 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.532106 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:05.532106 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.532106 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.532106 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:05.532106 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.532106 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:05.532601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.533442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:05.534210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:05.535076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:05.535789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:05.535857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:05.536237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.536960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:05.536960 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.536960 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:05.536960 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.536960 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.536960 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:05.536960 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.536960 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:05.538208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:05.538915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:05.538915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.538915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:05.538915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.538915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.538915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:05.538915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.538915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:05.539770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.540559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:05.541371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.541371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:05.541371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:05.541371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:05.541610 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:05.541954 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:05.542204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.543084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:05.543084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.543084 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:05.543084 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.543084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.543084 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:05.543084 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.543084 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.544779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:05.545320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:10.292171 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:10.293946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:10.325066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:10.325542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.326367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:10.326367 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.326367 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:10.326367 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.326367 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.326367 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:10.326367 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.326367 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.327210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.327582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:10.470879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:10.472195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:10.472511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:10.472747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.473675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:10.473675 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.473675 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:10.473675 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.473675 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.473675 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:10.473675 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.473675 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.475779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.476677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:10.529969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:10.530673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:10.531230 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:10.531518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.532361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:10.532361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.532361 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:10.532361 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.532361 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.532361 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:10.532361 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.532361 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:10.533760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:10.534678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:10.534678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.534678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:10.534678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.534678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.534678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:10.534678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.534678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:10.535726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.536532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.537357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:10.537357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.537357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:10.537357 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:10.537357 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:10.537357 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:10.537691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.538321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:10.538321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.538321 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:10.538321 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.538321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.538321 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:10.538321 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.538321 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:10.539712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:10.540478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:10.540478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.540478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:10.540478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.540478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.540478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:10.540478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.540478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:10.541625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.542586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:10.543440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.543440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:10.543440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:10.543440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:10.543440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:10.543440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:10.543440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.543440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:10.543911 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.543911 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:10.543911 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.543911 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.543911 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:10.543911 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.543911 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:10.545474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:15.293024 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:15.294324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:15.326789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:15.327263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.328156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:15.328156 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.328156 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:15.328156 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.328156 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.328156 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:15.328156 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.328156 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.329336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.330251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:15.471606 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:15.472788 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:15.473196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:15.473439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.474552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:15.474552 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.474552 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:15.474552 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.474552 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.474552 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:15.474552 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.474552 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.476211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.477023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:15.530583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:15.531173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:15.531696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:15.532004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.532760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:15.532760 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.532760 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:15.532760 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.532760 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.532760 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:15.532760 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.532760 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:15.534159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:15.534743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:15.534743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.534743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:15.534743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.534743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.534743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:15.534743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.534743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:15.535751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.536377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.536917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:15.536917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.536917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:15.536917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:15.536917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:15.537599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:15.537862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.538417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:15.538417 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.538417 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:15.538417 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.538417 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.538417 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:15.538417 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.538417 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:15.539079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:15.539614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:15.539614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.539614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:15.539614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.539614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.539614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:15.539614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.539614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:15.539830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.540319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:15.541009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.541009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:15.542344 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:15.542873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:15.543321 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:15.543321 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:15.543494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.544090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:15.544090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.544090 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:15.544090 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.544090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.544090 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:15.544090 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.544090 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.545809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:15.546513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:20.293726 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:20.296109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:20.330307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:20.330745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.331692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:20.331692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.331692 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:20.331692 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.331692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.331692 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:20.331692 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.331692 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.332626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.333327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:20.472562 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:20.473823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:20.474217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:20.474729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.475363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:20.475363 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.475363 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:20.475363 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.475363 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.475363 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:20.475363 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.475363 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.477180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.477960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:20.531328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:20.531871 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:20.532503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:20.532812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.533528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:20.533528 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.533528 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:20.533528 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.533528 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.533528 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:20.533528 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.533528 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:20.535070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:20.535706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:20.535706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.535706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:20.535706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.535706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.535706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:20.535706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.535706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:20.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.537260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.537871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:20.537871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.537871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:20.537871 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:20.537871 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:20.538108 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:20.538195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.538901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:20.538901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.538901 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:20.538901 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.538901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.538901 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:20.538901 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.538901 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:20.540576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:20.541112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:20.541112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.541112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:20.541112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.541112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.541112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:20.541112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.541112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:20.542761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.543370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:20.544461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.544461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:20.544461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:20.544461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:20.544461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:20.544841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:20.544841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.545700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:20.545700 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.545700 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:20.545700 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.545700 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.545700 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:20.545700 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.545700 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.547445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:20.547928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:25.294524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:25.296205 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:25.334357 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:25.334791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.335700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:25.335700 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.335700 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:25.335700 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.335700 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.335700 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:25.335700 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.335700 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.336923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.337846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:25.473252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:25.474633 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:25.474932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:25.474932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.475736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:25.475736 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.475736 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:25.475736 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.475736 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.475736 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:25.475736 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.475736 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.477387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.478289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:25.532077 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:25.532299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:25.533119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:25.533401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.534138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:25.534138 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.534138 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:25.534138 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.534138 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.534138 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:25.534138 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.534138 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:25.534959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:25.535868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:25.535868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.535868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:25.535868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.535868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.535868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:25.535868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.535868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:25.536283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.537009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:25.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:25.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:25.538084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:25.538516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:25.538733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.539251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:25.539251 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.539251 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:25.539251 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.539251 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.539251 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:25.539251 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.539251 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:25.539937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:25.540648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:25.540648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.540648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:25.540648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.540648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.540648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:25.540648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.540648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:25.541033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.541754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:25.542493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.542493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:25.543431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:25.543878 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:25.544139 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:25.544370 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:25.544518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.545181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:25.545181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.545181 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:25.545181 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.545181 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.545181 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:25.545181 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.545181 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.546370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:25.547484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:30.295299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:30.297069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:30.299229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=239 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:27:30.335503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:30.335872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.336553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:30.336553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.336553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:30.336553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.336553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.336553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:30.336553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.336553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.338019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.338891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:30.474061 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:30.475516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:30.475853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:30.476096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.477035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:30.477035 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.477035 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:30.477035 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.477035 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.477035 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:30.477035 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.477035 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.478543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.479636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:30.532541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:30.533053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:30.533622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:30.533908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.534632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:30.534632 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.534632 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:30.534632 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.534632 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.534632 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:30.534632 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.534632 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:30.535796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:30.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:30.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:30.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:30.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.536682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:30.537455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.538228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.539080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:30.539080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.539080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:30.539080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:30.539080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:30.539080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:30.539080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.539798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:30.539798 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.539798 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:30.539798 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.539798 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.539798 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:30.539798 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.539798 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:30.541073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:30.541737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:30.541737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.541737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:30.541737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.541737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.541737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:30.541737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.541737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:30.542284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.542927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:30.543444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.543444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:30.543855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:30.544509 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:30.544853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:30.545093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:30.545264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.545872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:30.545872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.545872 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:30.545872 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.545872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.545872 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:30.545872 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.545872 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.547498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:30.548004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:35.296198 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:35.297533 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:35.332186 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:35.332698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.333595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:35.333595 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.333595 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:35.333595 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.333595 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.333595 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:35.333595 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.333595 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.334468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.335505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:35.474830 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:35.475852 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:35.476052 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:35.476311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.476769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:35.476769 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.476769 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:35.476769 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.476769 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.476769 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:35.476769 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.476769 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.477532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.478487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:35.533113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:35.533513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:35.534059 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:35.534361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.535136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:35.535136 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.535136 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:35.535136 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.535136 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.535136 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:35.535136 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.535136 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:35.536524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:35.537065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:35.537065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.537065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:35.537065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.537065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.537065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:35.537065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.537065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:35.537573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.538057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.538577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:35.538577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.538577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:35.538577 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:35.538577 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:35.538989 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:35.539353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.540028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:35.540028 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.540028 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:35.540028 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.540028 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.540028 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:35.540028 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.540028 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:35.540747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:35.541199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:35.541199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.541199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:35.541199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.541199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.541199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:35.541199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.541199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:35.541702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.542189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:35.542690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.542690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:35.544211 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:35.544841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:35.545204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:35.545423 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:35.545565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.546141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:35.546141 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.546141 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:35.546141 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.546141 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.546141 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:35.546141 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.546141 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:35.548287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:40.296821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:40.298473 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:40.335186 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:40.335652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.336089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:40.336089 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.336089 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:40.336089 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.336089 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.336089 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:40.336089 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.336089 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.336900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.337697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:40.475786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:40.476778 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:40.477036 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:40.477185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.477882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:40.477882 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.477882 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:40.477882 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.477882 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.477882 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:40.477882 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.477882 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.479612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.480502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:40.533775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:40.534232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:40.534751 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:40.535132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.536102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:40.536102 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.536102 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:40.536102 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.536102 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.536102 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:40.536102 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.536102 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:40.537079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:40.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:40.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:40.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:40.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.537773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:40.538390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.538947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.539758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:40.539758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.539758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:40.539758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:40.539758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:40.539758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:40.540116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.540613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:40.540613 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.540613 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:40.540613 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.540613 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.540613 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:40.540613 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.540613 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:40.541417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:40.542093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:40.542093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.542093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:40.542093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.542093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.542093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:40.542093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.542093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:40.542721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.543227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:40.543758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.543758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:40.544639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:40.545084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:40.545440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:40.545696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:40.545882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.546451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:40.546451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.546451 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:40.546451 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.546451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.546451 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:40.546451 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.546451 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.547733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:40.548224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:45.297766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:45.299416 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:45.336587 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:45.336868 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.337631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:45.337631 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.337631 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:45.337631 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.337631 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.337631 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:45.337631 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.337631 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.338848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.340259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:45.476562 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:45.477922 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:45.478290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:45.478553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.479456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:45.479456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.479456 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:45.479456 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.479456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.479456 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:45.479456 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.479456 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.481337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.482200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:45.534345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:45.534869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:45.535866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:45.535866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.536615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:45.536615 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.536615 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:45.536615 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.536615 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.536615 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:45.536615 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.536615 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:45.537864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:45.538386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:45.538386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.538386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:45.538386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.538386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.538386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:45.538386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.538386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:45.539664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.540243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.541168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:45.541168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.541168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:45.541168 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:45.541168 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:45.541168 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:45.541516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.542240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:45.542240 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.542240 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:45.542240 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.542240 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.542240 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:45.542240 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.542240 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:45.543333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.543913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:45.544730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:45.545377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:45.545667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:45.546031 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:45.546316 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:45.546489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.547055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:27:45.547055 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.547055 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:45.547055 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.547055 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.547055 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:45.547055 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.547055 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.548319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:45.548767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:50.298604 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:50.300220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:50.334902 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:50.335353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.336495 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:50.336495 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.336495 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:50.336495 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.336495 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.336495 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:50.336495 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.336495 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.337810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.338846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:50.477323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:50.479025 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:50.479433 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:50.479687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.481543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:50.481543 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.481543 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:50.481543 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.481543 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.481543 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:50.481543 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.481543 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.482814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.483325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:50.534970 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:50.535382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:50.536201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:50.536535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.537471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:50.537471 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.537471 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:50.537471 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.537471 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.537471 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:50.537471 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.537471 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:50.538635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:50.539170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:50.539170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.539170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:50.539170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.539170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.539170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:50.539170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.539170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:50.539600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.540090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.540607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:50.540607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.540607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:50.540607 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:50.541111 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:50.541652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:50.541998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.542876 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:27:50.542876 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.542876 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:50.542876 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.542876 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.542876 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:50.542876 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.542876 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:50.544498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.545064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:50.546478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.546990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:50.547451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.547451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:50.547451 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:50.547451 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:50.547947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:50.548340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:50.548598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.549533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:27:50.549533 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.549533 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:50.549533 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.549533 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.549533 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:50.549533 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.549533 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.550851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:50.551346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:55.299268 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:55.301050 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:55.338112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:55.338398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.339152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:27:55.339152 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.339152 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:55.339152 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.339152 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.339152 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:55.339152 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.339152 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.340294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.341192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:55.478146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:55.479533 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:55.480132 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:55.480385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.481428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:55.481428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.481428 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:55.481428 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.481428 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.481428 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:55.481428 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.481428 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.483575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.484480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:55.535605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:55.535900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:27:55.536721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:55.537077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.537787 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:55.537787 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.537787 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:55.537787 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.537787 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.537787 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:55.537787 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.537787 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:55.538804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:55.539672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:55.539672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.539672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:55.539672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.539672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.539672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:55.539672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.539672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:27:55.540202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.540929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.541794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:55.541794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.541794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:55.541794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:55.541794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:55.541794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:55.542163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.542802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:27:55.542802 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.542802 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:55.542802 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.542802 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.542802 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:55.542802 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.542802 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:27:55.544288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.545337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:27:55.546078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.547384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:55.548683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.548683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:27:55.548683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:27:55.548683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:27:55.548683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:55.548683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:27:55.548683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.549142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:27:55.549142 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.549142 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:27:55.549142 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.549142 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.549142 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:27:55.549142 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.549142 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.550030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:27:55.550617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:00.300032 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:00.301331 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:00.336011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:00.336461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.337338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:00.337338 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.337338 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:00.337338 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.337338 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.337338 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:00.337338 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.337338 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.338227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.338689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:00.478769 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:00.480140 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:00.480459 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:00.480650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.481641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:00.481641 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.481641 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:00.481641 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.481641 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.481641 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:00.481641 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.481641 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.483439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.484575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:00.536311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:00.536620 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:00.537327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:00.537674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.538492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:00.538492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.538492 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:00.538492 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.538492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.538492 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:00.538492 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.538492 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:00.539707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.541432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:00.542556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:00.543135 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.543692 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:00.544717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:00.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.547029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:00.547561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.547561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:00.547561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:00.547905 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:00.548181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:00.548323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:00.548428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.548795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:00.548795 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.548795 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:00.548795 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.548795 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.548795 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:00.548795 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.548795 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.550119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:00.550453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:05.300573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:05.302222 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:05.335187 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:05.335422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.335994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:05.335994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.335994 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:05.335994 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.335994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.335994 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:05.335994 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.335994 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.337070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.337619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:05.479573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:05.480858 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:05.481772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:05.481772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.482029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:05.482029 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.482029 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:05.482029 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.482029 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.482029 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:05.482029 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.482029 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.483875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.484621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:05.536776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:05.537762 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:05.538354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:05.538647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.539501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:05.539501 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.539501 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:05.539501 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.539501 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.539501 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:05.539501 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.539501 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:05.540031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.540841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:05.541630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:05.542551 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:05.543710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:05.543902 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:05.544448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.544647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:05.544647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.544647 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:05.544647 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.544647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.544647 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:05.544647 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.544647 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:05.545448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.546134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:05.546543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.547267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:05.548101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.548101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:05.548438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:05.548656 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:05.548989 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:05.549228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:05.549378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.549832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:05.549832 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.549832 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:05.549832 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.549832 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.549832 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:05.549832 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.549832 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.551159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:05.552016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:10.301249 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:10.302495 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:10.342670 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:10.342930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.343627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:10.343627 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.343627 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:10.343627 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.343627 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.343627 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:10.343627 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.343627 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.344811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.346147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:10.480363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:10.481485 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:10.481835 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:10.482117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.483410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:10.483410 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.483410 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:10.483410 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.483410 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.483410 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:10.483410 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.483410 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.484404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.485540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:10.537465 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:10.537914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:10.538503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:10.538819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.539617 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:10.539617 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.539617 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:10.539617 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.539617 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.539617 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:10.539617 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.539617 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:10.540779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:10.541764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:10.541764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.541764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:10.541764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.541764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.541764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:10.541764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.541764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:10.542305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.543064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.543862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:10.543862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.543862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:10.543862 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:10.544146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:10.544524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:10.544873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.545622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:10.545622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.545622 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:10.545622 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.545622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.545622 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:10.545622 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.545622 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:10.546941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.547796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:10.548637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.549309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:10.549890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.549890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:10.549890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:10.549890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:10.549890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:10.550081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:10.550281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.550920 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:10.550920 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.550920 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:10.550920 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.550920 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.550920 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:10.550920 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.550920 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.552500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:10.553199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:15.301820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:15.303498 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:15.337450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:15.337717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.338427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:15.338427 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.338427 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:15.338427 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.338427 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.338427 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:15.338427 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.338427 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.339467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.340280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:15.481005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:15.482503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:15.482881 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:15.483196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.484058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:15.484058 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.484058 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:15.484058 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.484058 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.484058 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:15.484058 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.484058 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.486261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.487131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:15.538198 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:15.538639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:15.539533 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:15.540094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.540847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:15.540847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.540847 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:15.540847 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.540847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.540847 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:15.540847 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.540847 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:15.541792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:15.542612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:15.542612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.542612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:15.542612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.542612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.542612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:15.542612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.542612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:15.543258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.544190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.545008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:15.545008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.545008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:15.545008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:15.545008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:15.545008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:15.545319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.545866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:15.545866 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.545866 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:15.545866 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.545866 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.545866 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:15.545866 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.545866 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:15.547072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.547720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:15.548137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.549071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:15.549873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.549873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:15.549873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:15.549873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:15.549873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:15.550160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:15.550218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.550686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:15.550686 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.550686 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:15.550686 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.550686 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.550686 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:15.550686 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.550686 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.551623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:15.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:20.302658 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:20.303655 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:20.336105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:20.336443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.337158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:20.337158 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.337158 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:20.337158 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.337158 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.337158 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:20.337158 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.337158 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.337935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.338330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:20.481915 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:20.483335 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:20.483655 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:20.483885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.484816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:20.484816 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.484816 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:20.484816 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.484816 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.484816 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:20.484816 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.484816 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.486641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.487466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:20.538900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:20.539391 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:20.540071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:20.540382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.541226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:20.541226 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.541226 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:20.541226 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.541226 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.541226 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:20.541226 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.541226 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:20.542419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:20.543033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:20.543033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.543033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:20.543033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.543033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.543033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:20.543033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.543033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:20.543871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.544319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.544792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:20.544792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.544792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:20.544792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:20.544792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:20.545126 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:20.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.546360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:20.546360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.546360 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:20.546360 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.546360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.546360 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:20.546360 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.546360 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:20.547601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.548016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:20.549336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.549802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:20.550294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.550294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:20.550294 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:20.550294 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:20.550662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:20.550987 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:20.551239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.551885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:20.551885 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.551885 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:20.551885 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.551885 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.551885 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:20.551885 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.551885 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.552990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:20.553466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:24.807587 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:24.819668 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 26 01:28:24.819877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105681) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}}
Mar 26 01:28:24.820113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:28:24.857345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, resource={'id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'name': 'private', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['02d96a4b-4c93-4b95-ad0f-f11d085f0699', '0503b55a-1b69-49b5-b418-16ba37be90e9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T00:58:10Z', 'updated_at': '2026-03-26T00:58:13Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4449}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.857740 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.858436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6fe31530-ff9f-476b-aea1-4804059b76cd, external_ids_update=None, columns={'tag': 4449}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.858678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.859534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:28:24.859534 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:24.859534 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:24.859534 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:24.859534 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:24.859534 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:24.859534 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.859534 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:24.860515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:28:24.861104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:24.861707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:28:24.861707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.861707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:28:24.861707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:28:24.891855 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-db438906-c74f-4dc8-883c-6e56962e551b, resource={'id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'name': 'public', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['1c3e4583-6e63-4953-8050-3341662a5ec6', '23682989-62bf-4e07-a74f-0cb01077d5ac'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-26T00:58:16Z', 'updated_at': '2026-03-26T00:58:25Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.892398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-db438906-c74f-4dc8-883c-6e56962e551b, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.893051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-200618ec-0540-41b0-9429-dc7dfc74abb0, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.893413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.894297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:28:24.894297 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:24.894297 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:24.894297 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:24.894297 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:24.894297 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:24.894297 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.894297 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:24.895313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:28:24.895795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:24.896395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:28:24.896395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.896395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:28:24.896395 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:28:24.924994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, resource={'id': 'f7e819e4-7101-412b-b047-baa5e16b2a52', 'name': 'admin_net', 'tenant_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'project_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f2e900ee-4d88-4151-8e50-661745950632'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:14Z', 'updated_at': '2026-03-26T01:02:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4803}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.925449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.925992 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d43d91f1-f24a-411f-bae7-af85048f95b9, external_ids_update=None, columns={'tag': 4803}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.926183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.927007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:28:24.927007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:24.927007 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:24.927007 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:24.927007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:24.927007 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:24.927007 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.927007 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:24.927862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:28:24.928200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:24.928673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:28:24.928673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.928673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:28:24.928673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:28:24.958722 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, resource={'id': 'f481c2f9-5052-46d5-821a-6c8712ec70ec', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['329ea1d3-f656-45b1-bf23-1d927d4578ea'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:50Z', 'updated_at': '2026-03-26T01:02:51Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19903}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.959251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.959859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3aa5be7a-5c67-4f1a-bdf2-7bf6912e871d, external_ids_update=None, columns={'tag': 19903}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:24.960139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.961107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:28:24.961107 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:24.961107 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:24.961107 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:24.961107 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:24.961107 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:24.961107 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.961107 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:24.962342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:28:24.962688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:24.963108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:28:24.963108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:24.963108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:28:24.963108 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:28:25.027652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, resource={'id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'enable_snat': True}, 'gw_port_id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-ryt6n', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'ip_address': '172.24.5.142', 'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac'}, {'ip_address': '2001:db8::16a', 'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 3, 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8'}, resource_type=routers, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.027928 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.028170 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.028634 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.028634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.029323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:28:25.029323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.029323 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:25.029323 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.029323 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.029323 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:25.029323 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.029323 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:25.029710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Unable to update router e4efd798-3c25-4ea9-bd45-fa7a05aeaef9. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 26 01:28:25.030615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:28:25.031176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:28:25.031747 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Fixing resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:28:25.115618 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, resource={'id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'name': '', 'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'project_id': '', 'mac_address': 'fa:16:3e:4e:d4:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160849', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.115915 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::16a/64', '172.24.5.142/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.116424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=14a0aa42-40aa-4508-9af3-cffdffef9818, lrouter_port=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.116779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.117661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Traceback (most recent call last):
Mar 26 01:28:25.117661 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.117661 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:25.117661 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.117661 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.117661 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:25.117661 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.117661 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Failed to fix resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:25.118926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.119331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.119706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:28:25.119706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.119706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:28:25.119706 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1bd9db42-79df-4bf8-b405-b0a4e1c1eb53 None None] Maintenance task: Synchronization completed (took 0.30 seconds)
Mar 26 01:28:25.303208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:25.304695 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:25.339354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.339663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.340333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:25.340333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.340333 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:25.340333 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.340333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.340333 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:25.340333 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.340333 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.341583 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.342513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:25.482646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:25.483820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.484283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.484502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.485510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:25.485510 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.485510 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:25.485510 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.485510 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.485510 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:25.485510 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.485510 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.487354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.488156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:25.539583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:25.540009 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:25.540643 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.540995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.541728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:25.541728 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.541728 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:25.541728 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.541728 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.541728 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:25.541728 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.541728 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:25.542593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.543387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:25.543387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.543387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:25.543387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.543387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.543387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:25.543387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.543387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:25.543737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.544400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.545206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:25.545206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.545206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:25.545206 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:25.545206 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:25.545649 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.545910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.546448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:25.546448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.546448 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:25.546448 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.546448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.546448 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:25.546448 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.546448 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:25.547870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.548288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:25.549760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.550227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:25.550776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.550776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:25.550776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:25.550776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:25.550776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.550956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:25.551118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.551717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:25.551717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.551717 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:25.551717 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.551717 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.551717 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:25.551717 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.551717 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.552997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:25.553546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:30.304031 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:30.306274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:30.308511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=434 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:28:30.344350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:30.345023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.345901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:30.345901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.345901 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:30.345901 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.345901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.345901 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:30.345901 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.345901 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.346763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.347411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:30.483358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:30.484686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:30.484826 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:30.484955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.485742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:30.485742 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.485742 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:30.485742 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.485742 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.485742 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:30.485742 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.485742 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.488302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.489427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:30.540274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:30.540765 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:30.541482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:30.541710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.542314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:30.542314 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.542314 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:30.542314 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.542314 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.542314 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:30.542314 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.542314 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:30.543596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:30.544149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:30.544149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.544149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:30.544149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.544149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.544149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:30.544149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.544149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:30.545358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.545844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.546355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:30.546355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.546355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:30.546355 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:30.546355 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:30.546665 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:30.546892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.547479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:30.547479 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.547479 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:30.547479 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.547479 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.547479 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:30.547479 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.547479 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:30.548919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.549377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:30.550872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.551369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:30.551888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.551888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:30.551888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:30.551888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:30.552108 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:30.552333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:30.552504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.553090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:30.553090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.553090 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:30.553090 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.553090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.553090 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:30.553090 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.553090 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.554353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:30.554831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:35.304777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:35.306253 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:35.340935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:35.341269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.341998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:35.341998 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.341998 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:35.341998 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.341998 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.341998 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:35.341998 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.341998 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.342849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.343563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:35.483931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:35.485680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:35.486084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:35.486345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.487445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:35.487445 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.487445 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:35.487445 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.487445 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.487445 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:35.487445 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.487445 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.489550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.490414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:35.540936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:35.541526 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:35.542407 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:35.542655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.543449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:35.543449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.543449 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:35.543449 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.543449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.543449 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:35.543449 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.543449 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:35.544440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.545675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:35.546367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:35.547190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.547954 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:35.548886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:35.549701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:35.549701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.549701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:35.549701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.549701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.549701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:35.549701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.549701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:35.550118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.550823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:35.551660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.551660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:35.551877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:35.552358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:35.552780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:35.553323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:35.553511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.554055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:35.554055 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.554055 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:35.554055 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.554055 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.554055 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:35.554055 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.554055 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.555183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:35.555657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:40.305432 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:40.306616 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:40.337461 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:40.337749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.338477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:40.338477 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.338477 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:40.338477 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.338477 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.338477 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:40.338477 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.338477 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.339661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.340634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:40.484718 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:40.485804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:40.486389 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:40.486389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.487392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:40.487392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.487392 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:40.487392 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.487392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.487392 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:40.487392 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.487392 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.489093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.490743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:40.541547 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:40.542254 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:40.542675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:40.542906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.543424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:40.543424 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.543424 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:40.543424 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.543424 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.543424 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:40.543424 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.543424 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:40.545563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:40.546063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:40.546063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.546063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:40.546063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.546063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.546063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:40.546063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.546063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:40.546831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.547299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.547962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:40.547962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.547962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:40.547962 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:40.548128 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:40.548726 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:40.549281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.550256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:40.550256 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.550256 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:40.550256 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.550256 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.550256 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:40.550256 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.550256 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:40.551322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:40.551737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:40.551737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.551737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:40.551737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.551737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.551737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:40.551737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.551737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:40.552467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.553236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:40.553727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.553727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:40.554039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:40.554290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:40.554705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:40.554890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:40.555059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.555580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:40.555580 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.555580 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:40.555580 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.555580 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.555580 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:40.555580 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.555580 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.556652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:40.557218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:45.306126 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:45.307766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:45.338300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:45.338859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.339497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:45.339497 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.339497 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:45.339497 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.339497 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.339497 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:45.339497 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.339497 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.340345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.340903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:45.485484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:45.486935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:45.487337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:45.487715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.488689 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:45.488689 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.488689 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:45.488689 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.488689 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.488689 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:45.488689 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.488689 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.490831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.491952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:45.542521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:45.543182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:45.543828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:45.544101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.544703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:45.544703 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.544703 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:45.544703 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.544703 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.544703 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:45.544703 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.544703 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:45.546178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:45.547081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:45.547081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.547081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:45.547081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.547081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.547081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:45.547081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.547081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:45.547727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.548565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.549436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:45.549436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.549436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:45.549436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:45.549436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:45.549436 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:45.549436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.549797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:45.549797 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.549797 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:45.549797 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.549797 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.549797 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:45.549797 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.549797 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:45.550855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:45.551549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:45.551549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.551549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:45.551549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.551549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.551549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:45.551549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.551549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:45.552488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.553023 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:45.553497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.553497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:45.554318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:45.554800 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:45.555203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:45.555563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:45.555791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.556593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:45.556593 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.556593 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:45.556593 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.556593 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.556593 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:45.556593 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.556593 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.558223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:45.558744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:50.306786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:50.308349 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:50.343244 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:50.343633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.344481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:50.344481 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.344481 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:50.344481 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.344481 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.344481 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:50.344481 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.344481 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.345451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.346043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:50.486163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:50.487508 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:50.487850 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:50.488122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.489050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:50.489050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.489050 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:50.489050 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.489050 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.489050 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:50.489050 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.489050 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.490854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.491748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:50.543078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:50.543576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:50.544240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:50.544614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.545491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:50.545491 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.545491 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:50.545491 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.545491 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.545491 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:50.545491 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.545491 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:50.547028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:50.547802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:50.547802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.547802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:50.547802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.547802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.547802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:50.547802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.547802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:50.549706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.550461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.551264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:50.551264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.551264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:50.551264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:50.551264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:50.551264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:50.551264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.551854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:50.551854 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.551854 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:50.551854 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.551854 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.551854 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:50.551854 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.551854 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:50.552566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:50.553257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:50.553257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.553257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:50.553257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.553257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.553257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:50.553257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.553257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:50.553594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.554310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:50.555064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.555064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:50.555064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:50.555064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:50.555484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:50.555798 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:50.556051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.556700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:50.556700 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.556700 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:50.556700 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.556700 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.556700 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:50.556700 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.556700 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.557713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:50.558242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:55.307524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:55.309109 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:55.340743 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:55.341079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.341749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:28:55.341749 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.341749 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:55.341749 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.341749 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.341749 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:55.341749 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.341749 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.342991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.343565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:55.486839 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:55.487913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:55.488260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:55.488476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.489290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:55.489290 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.489290 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:55.489290 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.489290 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.489290 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:55.489290 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.489290 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.490496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.491119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:55.543581 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:55.544002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:28:55.544733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:55.544848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.545727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:55.545727 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.545727 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:55.545727 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.545727 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.545727 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:55.545727 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.545727 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:55.546897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:55.548159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:55.548159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.548159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:55.548159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.548159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.548159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:55.548159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.548159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:28:55.548652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.549607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.550806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:55.550806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.550806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:55.550806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:55.551105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:55.551577 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:55.551914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.552629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:28:55.552629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.552629 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:55.552629 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.552629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.552629 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:55.552629 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.552629 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:28:55.553485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:55.553910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:28:55.553910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.553910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:28:55.553910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.553910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.553910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:28:55.553910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.553910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:28:55.555295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.555917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:55.556586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.556586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:28:55.556586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:28:55.556586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:28:55.556586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:55.556923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:28:55.557212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.558015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:28:55.558015 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.558015 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:28:55.558015 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.558015 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.558015 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:28:55.558015 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.558015 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.559284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:28:55.559852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:00.308183 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:00.309331 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:00.341542 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:00.342146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.343087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:00.343087 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.343087 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:00.343087 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.343087 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.343087 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:00.343087 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.343087 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.343950 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.345027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:00.487482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:00.488872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:00.489202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:00.489444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.490332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:00.490332 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.490332 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:00.490332 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.490332 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.490332 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:00.490332 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.490332 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.492196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.493512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:00.544320 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:00.544699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:00.545260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:00.545580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.546696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:00.546696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.546696 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:00.546696 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.546696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.546696 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:00.546696 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.546696 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:00.548209 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:00.549353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:00.549353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.549353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:00.549353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.549353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.549353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:00.549353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.549353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:00.549738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.550478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.551448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:00.551448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.551448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:00.551448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:00.551679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:00.552007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:00.552388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.553140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:00.553140 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.553140 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:00.553140 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.553140 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.553140 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:00.553140 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.553140 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:00.554429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:00.555213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:00.555213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.555213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:00.555213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.555213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.555213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:00.555213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.555213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:00.556154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.556639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:00.557166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.557166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:00.557166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:00.557166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:00.557166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:00.557443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:00.557580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.558078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:00.558078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.558078 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:00.558078 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.558078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.558078 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:00.558078 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.558078 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.559185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:00.559696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:05.308783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:05.310248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:05.346687 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:05.347105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.348078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:05.348078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.348078 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:05.348078 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.348078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.348078 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:05.348078 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.348078 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.349548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.350336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:05.488348 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:05.489343 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:05.489503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:05.489591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.490101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:05.490101 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.490101 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:05.490101 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.490101 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.490101 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:05.490101 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.490101 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.491731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.492218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:05.544829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:05.545266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:05.546037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:05.546375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.547210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:05.547210 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.547210 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:05.547210 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.547210 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.547210 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:05.547210 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.547210 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:05.548569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:05.549325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:05.549325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.549325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:05.549325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.549325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.549325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:05.549325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.549325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:05.550266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.551141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.551733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:05.551733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.551733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:05.551733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:05.552114 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:05.552631 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:05.553019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.553745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:05.553745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.553745 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:05.553745 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.553745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.553745 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:05.553745 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.553745 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:05.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:05.555902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:05.555902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.555902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:05.555902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.555902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.555902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:05.555902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.555902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:05.556755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.557703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:05.558714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.558714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:05.558714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:05.558714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:05.558714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:05.558714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:05.559069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.559545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:05.559545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.559545 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:05.559545 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.559545 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.559545 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:05.559545 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.559545 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.561849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:05.562685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:10.309480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:10.311172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:10.348689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:10.348998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.349839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:10.349839 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.349839 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:10.349839 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.349839 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.349839 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:10.349839 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.349839 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.350744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.351277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:10.488847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:10.490152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:10.490565 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:10.490800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.491659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:10.491659 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.491659 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:10.491659 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.491659 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.491659 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:10.491659 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.491659 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.493480 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.494020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:10.545327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:10.545817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:10.546476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:10.546782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.547520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:10.547520 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.547520 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:10.547520 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.547520 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.547520 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:10.547520 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.547520 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:10.548800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:10.549652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:10.549652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.549652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:10.549652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.549652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.549652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:10.549652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.549652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:10.550283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.551037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.551829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:10.551829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.551829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:10.552293 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:10.552744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:10.553256 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:10.553612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.554513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:10.554513 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.554513 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:10.554513 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.554513 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.554513 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:10.554513 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.554513 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:10.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:10.556493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:10.556493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.556493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:10.556493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.556493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.556493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:10.556493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.556493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:10.557379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.557844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:10.558381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.558381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:10.558381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:10.558381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:10.558512 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:10.558669 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:10.558805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.559330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:10.559330 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.559330 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:10.559330 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.559330 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.559330 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:10.559330 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.559330 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.560432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:15.310033 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:15.311549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:15.349690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:15.350272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.351161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:15.351161 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.351161 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:15.351161 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.351161 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.351161 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:15.351161 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.351161 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.352423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.352902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:15.489521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:15.490914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:15.491366 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:15.491677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.492803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:15.492803 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.492803 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:15.492803 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.492803 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.492803 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:15.492803 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.492803 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.494665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.495492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:15.545946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:15.546491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:15.546973 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:15.547217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.547822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:15.547822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.547822 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:15.547822 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.547822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.547822 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:15.547822 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.547822 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:15.549574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:15.550500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:15.550500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.550500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:15.550500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.550500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.550500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:15.550500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.550500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:15.551487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.552184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.552945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:15.552945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.552945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:15.552945 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:15.553261 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:15.553541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:15.553709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.554076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:15.554076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.554076 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:15.554076 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.554076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.554076 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:15.554076 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.554076 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:15.554723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.555415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:15.556003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.556520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:15.558031 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:15.558248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:15.558579 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:15.558850 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:15.559048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.559571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:15.559571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.559571 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:15.559571 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.559571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.559571 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:15.559571 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.559571 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.560705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:15.561195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:20.310668 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:20.312357 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:20.343458 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:20.343677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.344235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:20.344235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.344235 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:20.344235 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.344235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.344235 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:20.344235 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.344235 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.345339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.346516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:20.490361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:20.492000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:20.492424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:20.492699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.494003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:20.494003 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.494003 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:20.494003 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.494003 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.494003 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:20.494003 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.494003 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.496121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.497467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:20.546584 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:20.546914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:20.547476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:20.547779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.548539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:20.548539 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.548539 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:20.548539 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.548539 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.548539 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:20.548539 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.548539 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:20.549866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:20.550709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:20.550709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.550709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:20.550709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.550709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.550709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:20.550709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.550709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:20.551461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.552280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.553118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:20.553118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.553118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:20.553118 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:20.553914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:20.554456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:20.554816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.555800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:20.555800 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.555800 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:20.555800 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.555800 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.555800 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:20.555800 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.555800 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:20.557515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:20.557966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:20.557966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.557966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:20.557966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.557966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.557966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:20.557966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.557966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:20.559807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.560337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:20.560820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.560820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:20.560820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:20.560820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:20.560820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:20.561204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:20.561459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.562357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:20.562357 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.562357 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:20.562357 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.562357 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.562357 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:20.562357 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.562357 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.564054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:20.564578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:25.311373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:25.312959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:25.349392 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:25.349805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.350721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:25.350721 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.350721 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:25.350721 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.350721 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.350721 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:25.350721 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.350721 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.351998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.352927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:25.491074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:25.492181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:25.492534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:25.492782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.493574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:25.493574 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.493574 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:25.493574 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.493574 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.493574 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:25.493574 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.493574 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.494732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.495351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:25.547184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:25.547695 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:25.548912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.549934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:25.549934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.549934 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:25.549934 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.549934 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.549934 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:25.549934 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.549934 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:25.551339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:25.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:25.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:25.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:25.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.552207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:25.552954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.553703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.554625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:25.554625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.554625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:25.554625 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:25.554625 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:25.555068 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:25.555484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.556448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:25.556448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.556448 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:25.556448 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.556448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.556448 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:25.556448 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.556448 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:25.557823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:25.558859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:25.558859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.558859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:25.558859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.558859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.558859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:25.558859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.558859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:25.559621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.560517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:25.561446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.561446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:25.561446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:25.561446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:25.561446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:25.561446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:25.561446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.561848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:25.561848 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.561848 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:25.561848 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.561848 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.561848 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:25.561848 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.561848 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.563191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:25.564143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:30.312099 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:30.313722 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:30.315553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:29:30.351411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:30.351831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.352750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:30.352750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.352750 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:30.352750 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.352750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.352750 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:30.352750 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.352750 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.353804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.354762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:30.491776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:30.492947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:30.493218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:30.493415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.494153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:30.494153 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.494153 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:30.494153 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.494153 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.494153 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:30.494153 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.494153 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.495790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.496619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:30.548190 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:30.548750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:30.549390 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:30.549743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.550564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:30.550564 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.550564 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:30.550564 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.550564 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.550564 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:30.550564 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.550564 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:30.551857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:30.552743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:30.552743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.552743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:30.552743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.552743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.552743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:30.552743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.552743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:30.553349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.554237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.555268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:30.555268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.555268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:30.555268 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:30.555268 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:30.555497 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:30.555878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.556709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:30.556709 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.556709 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:30.556709 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.556709 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.556709 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:30.556709 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.556709 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:30.557968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:30.558608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:30.558608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.558608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:30.558608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.558608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.558608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:30.558608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.558608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:30.559785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.560347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:30.560979 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.560979 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:30.560979 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:30.561127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:30.561421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:30.561666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:30.561837 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.562501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:30.562501 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.562501 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:30.562501 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.562501 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.562501 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:30.562501 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.562501 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.563805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:30.564364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:35.312810 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:35.314477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:35.346309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:35.346446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.346955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:35.346955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.346955 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:35.346955 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.346955 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.346955 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:35.346955 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.346955 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.347770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.348695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:35.492456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:35.493503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:35.493833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:35.494112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.494908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:35.494908 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.494908 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:35.494908 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.494908 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.494908 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:35.494908 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.494908 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.496101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.496850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:35.548766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:35.549334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:35.550085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:35.550312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.551316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:35.551316 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.551316 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:35.551316 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.551316 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.551316 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:35.551316 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.551316 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:35.552653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:35.553486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:35.553486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.553486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:35.553486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.553486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.553486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:35.553486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.553486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:35.554386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.555148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.555930 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:35.556539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.557254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:35.557637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.558423 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:35.558967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.558967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:35.561049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:35.561507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:35.561781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:35.561944 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:35.562061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.562449 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:35.562449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.562449 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:35.562449 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.562449 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.562449 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:35.562449 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.562449 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.563375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:35.563869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:40.313633 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:40.315300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:40.348755 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:40.349049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.349736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:40.349736 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.349736 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:40.349736 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.349736 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.349736 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:40.349736 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.349736 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.350965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.351742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:40.493760 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:40.494296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:40.494645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:40.494934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.496577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:40.496577 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.496577 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:40.496577 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.496577 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.496577 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:40.496577 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.496577 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.498021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.498908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:40.549262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:40.549693 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:40.550362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:40.550676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.551451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:40.551451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.551451 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:40.551451 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.551451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.551451 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:40.551451 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.551451 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:40.552649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:40.554151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:40.554151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.554151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:40.554151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.554151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.554151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:40.554151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.554151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:40.554640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.556033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.556969 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:40.558266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:40.559116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:40.559767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.560627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:40.561872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:40.562309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:40.562589 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:40.562798 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:40.562940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.563437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:40.563437 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.563437 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:40.563437 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.563437 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.563437 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:40.563437 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.563437 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.564691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:40.565086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:45.314348 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:45.316380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:45.348779 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:45.349238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.350221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:45.350221 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.350221 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:45.350221 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.350221 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.350221 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:45.350221 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.350221 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.351349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.352355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:45.493716 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:45.494898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:45.495256 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:45.495501 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.496405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:45.496405 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.496405 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:45.496405 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.496405 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.496405 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:45.496405 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.496405 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.498172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.499016 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:45.549799 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:45.550112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:45.550664 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:45.550998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.551861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:45.551861 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.551861 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:45.551861 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.551861 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.551861 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:45.551861 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.551861 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:45.553130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:45.553967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:45.553967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.553967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:45.553967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.553967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.553967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:45.553967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.553967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:45.554770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.555535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.556431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:45.556431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.556431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:45.556431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:45.556431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:45.556431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:45.556767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.557194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:45.557194 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.557194 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:45.557194 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.557194 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.557194 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:45.557194 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.557194 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:45.557874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.558624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:45.559398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.560362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:45.562356 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:45.562705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:45.563054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:45.563293 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:45.563444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.563940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:45.563940 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.563940 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:45.563940 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.563940 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.563940 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:45.563940 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.563940 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.565001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:45.565608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:50.315420 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:50.317044 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:50.352198 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:50.352458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.352800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:50.352800 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.352800 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:50.352800 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.352800 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.352800 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:50.352800 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.352800 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.353646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.355269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:50.494470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:50.495872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:50.496295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:50.496570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.497497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:50.497497 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.497497 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:50.497497 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.497497 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.497497 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:50.497497 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.497497 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.499200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.500171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:50.550336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:50.550803 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:50.551543 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:50.551913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.552693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:50.552693 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.552693 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:50.552693 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.552693 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.552693 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:50.552693 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.552693 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:50.553815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.555187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:50.556494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:50.557026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:50.557406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:50.557406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.558098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:50.558098 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.558098 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:50.558098 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.558098 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.558098 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:50.558098 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.558098 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:50.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:50.559094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:50.559094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.559094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:50.559094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.559094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.559094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:50.559094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.559094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:50.559327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.559881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:50.560438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.560438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:50.562732 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:50.563257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:50.563622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:50.563931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:50.564153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.564754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:29:50.564754 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.564754 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:50.564754 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.564754 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.564754 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:50.564754 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.564754 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.566661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:50.567191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:55.316212 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:55.317491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:55.348410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:55.348803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.349658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:29:55.349658 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.349658 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:55.349658 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.349658 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.349658 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:55.349658 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.349658 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.350591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.351765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:55.495117 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:55.496248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:55.496606 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:55.496860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.498033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:55.498033 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.498033 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:55.498033 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.498033 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.498033 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:55.498033 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.498033 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.499487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.500319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:55.551163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:55.551699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:29:55.552313 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:55.552624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.553524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:55.553524 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.553524 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:55.553524 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.553524 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.553524 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:55.553524 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.553524 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:55.555708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:55.556663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:55.556663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.556663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:55.556663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.556663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.556663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:55.556663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.556663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:29:55.557444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.558258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.559228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:55.559228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.559228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:55.559228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:55.559228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:55.559491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:55.559739 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.560518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:55.560518 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.560518 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:55.560518 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.560518 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.560518 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:55.560518 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.560518 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:29:55.561803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:55.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:29:55.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:29:55.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:29:55.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:29:55.563484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.564461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:29:55.565561 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.566154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.566154 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:29:55.566154 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.566154 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.566503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:29:55.567047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:00.316853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:00.317863 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:00.353155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:00.353498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.354414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:00.354414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.354414 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:00.354414 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.354414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.354414 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:00.354414 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.354414 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.355296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.356221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:00.495720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:00.497188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:00.497704 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:00.497704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.498587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:00.498587 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.498587 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:00.498587 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.498587 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.498587 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:00.498587 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.498587 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.500553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.501418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:00.551824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:00.552393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:00.553169 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:00.553465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.554275 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:00.554275 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.554275 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:00.554275 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.554275 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.554275 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:00.554275 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.554275 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:00.555654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:00.556616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:00.556616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.556616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:00.556616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.556616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.556616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:00.556616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.556616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:00.557234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.558069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.559246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:00.559246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.559246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:00.559246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:00.559246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:00.559246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:00.559559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.560333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:00.560333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.560333 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:00.560333 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.560333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.560333 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:00.560333 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.560333 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:00.561921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:00.562679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:00.562679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.562679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:00.562679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.562679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.562679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:00.562679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.562679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:00.563328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.563836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:00.564409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.564409 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:00.564409 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:00.564928 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:00.565383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:00.565714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:00.565934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.566731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:00.566731 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.566731 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:00.566731 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.566731 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.566731 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:00.566731 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.566731 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.568533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:00.569099 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:05.317455 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:05.319110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:05.352605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:05.353019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.353966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:05.353966 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.353966 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:05.353966 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.353966 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.353966 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:05.353966 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.353966 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.355846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.356678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:05.496696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:05.497808 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:05.498064 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:05.498177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.498750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:05.498750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.498750 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:05.498750 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.498750 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.498750 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:05.498750 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.498750 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.500523 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.501657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:05.552518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:05.553043 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:05.553422 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:05.553576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.554001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:05.554001 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.554001 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:05.554001 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.554001 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.554001 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:05.554001 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.554001 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:05.555017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:05.555967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:05.555967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.555967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:05.555967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.555967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.555967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:05.555967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.555967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:05.556416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.557179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.558066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:05.558066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.558066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:05.558549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:05.558940 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:05.559246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:05.559410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.559751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:05.559751 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.559751 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:05.559751 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.559751 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.559751 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:05.559751 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.559751 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:05.560271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.561055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:05.561553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.562221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:05.564610 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:05.565009 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:05.565280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:05.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:05.565605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.566065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:05.566065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.566065 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:05.566065 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.566065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.566065 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:05.566065 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.566065 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.566828 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:05.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:10.318217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:10.320678 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:10.352918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:10.353347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.354161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:10.354161 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.354161 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:10.354161 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.354161 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.354161 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:10.354161 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.354161 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.355169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.356091 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:10.497242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:10.498304 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:10.498576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:10.498797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.499570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:10.499570 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.499570 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:10.499570 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.499570 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.499570 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:10.499570 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.499570 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.501151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.502043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:10.553202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:10.553528 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:10.554232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:10.554483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.555085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:10.555085 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.555085 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:10.555085 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.555085 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.555085 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:10.555085 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.555085 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:10.556004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:10.556686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:10.556686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.556686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:10.556686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.556686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.556686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:10.556686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.556686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:10.557083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.557745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.558897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:10.558897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.558897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:10.559082 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:10.559340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:10.559832 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:10.560240 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.560905 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:10.561679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:10.562660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:10.562660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.562660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:10.562660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.562660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.562660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:10.562660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.562660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:10.563077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.563654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:10.564172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.564172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:10.564955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:10.565694 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:10.566137 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:10.566352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:10.566486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.566966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:10.566966 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.566966 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:10.566966 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.566966 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.566966 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:10.566966 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.566966 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.567915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:10.568491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:15.319015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:15.320446 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:15.353968 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:15.354520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.355432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:15.355432 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.355432 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:15.355432 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.355432 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.355432 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:15.355432 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.355432 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.356801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.357690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:15.497828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:15.498942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:15.499255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:15.499475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.500284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:15.500284 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.500284 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:15.500284 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.500284 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.500284 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:15.500284 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.500284 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.501726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.502185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:15.553614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:15.554582 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:15.554988 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:15.555238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.555811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:15.555811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.555811 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:15.555811 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.555811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.555811 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:15.555811 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.555811 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:15.556479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:15.557360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:15.557360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.557360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:15.557360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.557360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.557360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:15.557360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.557360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:15.557678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.558365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.559131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:15.559131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.559131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:15.559373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:15.559899 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:15.560290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:15.560505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.561172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:15.561172 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.561172 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:15.561172 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.561172 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.561172 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:15.561172 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.561172 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:15.562614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:15.563628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:15.563628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.563628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:15.563628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.563628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.563628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:15.563628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.563628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:15.564753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.565467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:15.565969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.565969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:15.565969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:15.565969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:15.565969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:15.566161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:15.566292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.566714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:15.566714 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.566714 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:15.566714 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.566714 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.566714 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:15.566714 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.566714 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.567688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:15.568221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:20.319558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:20.321010 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:20.355224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:20.355607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.356536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:20.356536 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.356536 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:20.356536 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.356536 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.356536 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:20.356536 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.356536 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.357941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.358809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:20.498571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:20.499949 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:20.500309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:20.500544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.501755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:20.501755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.501755 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:20.501755 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.501755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.501755 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:20.501755 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.501755 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.503674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.504183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:20.554829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:20.555094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:20.555585 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:20.556069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.556553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:20.556553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.556553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:20.556553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.556553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.556553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:20.556553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.556553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:20.558005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:20.558404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:20.558404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.558404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:20.558404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.558404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.558404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:20.558404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.558404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:20.559499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.559817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.560173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:20.560173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.560173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:20.560173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:20.560173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:20.560632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:20.561282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.562227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:20.562227 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.562227 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:20.562227 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.562227 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.562227 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:20.562227 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.562227 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:20.563373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:20.563762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:20.563762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.563762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:20.563762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.563762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.563762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:20.563762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.563762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:20.564858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.565197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:20.565618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.565618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:20.565665 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:20.566149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:20.566831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:20.566831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:20.567131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.567451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:20.567451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.567451 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:20.567451 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.567451 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.567451 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:20.567451 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.567451 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.569049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:20.570005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:25.320051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:25.321574 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:25.358386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:25.358725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.359515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:25.359515 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.359515 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:25.359515 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.359515 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.359515 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:25.359515 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.359515 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.360733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.361425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:25.499428 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:25.500551 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:25.500737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:25.500888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.501569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:25.501569 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.501569 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:25.501569 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.501569 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.501569 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:25.501569 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.501569 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.503357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.503797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:25.555351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:25.555612 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:25.556530 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:25.556530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.557145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:25.557145 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.557145 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:25.557145 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.557145 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.557145 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:25.557145 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.557145 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:25.558391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:25.559194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:25.559194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.559194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:25.559194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.559194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.559194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:25.559194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.559194 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:25.559885 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.560638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.561504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:25.561504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.561504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:25.561504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:25.561504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:25.561504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:25.561504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.562283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:25.562283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.562283 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:25.562283 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.562283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.562283 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:25.562283 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.562283 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:25.562880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:25.563487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:25.563487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.563487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:25.563487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.563487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.563487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:25.563487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.563487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:25.563769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.564434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:25.565093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.565093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:25.566057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:25.566521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:25.567056 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:25.567352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:25.567587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.568354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:25.568354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.568354 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:25.568354 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.568354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.568354 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:25.568354 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.568354 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.569458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:25.570226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:30.320791 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:30.322850 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:30.324428 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:30:30.355480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:30.355804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.356745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:30.356745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.356745 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:30.356745 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.356745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.356745 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:30.356745 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.356745 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.357547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.358437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:30.500129 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:30.501283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:30.501545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:30.501728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.502502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:30.502502 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.502502 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:30.502502 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.502502 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.502502 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:30.502502 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.502502 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.504264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.505160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:30.555853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:30.556164 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:30.556719 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:30.557079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.557824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:30.557824 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.557824 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:30.557824 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.557824 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.557824 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:30.557824 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.557824 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:30.559356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:30.560278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:30.560278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.560278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:30.560278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.560278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.560278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:30.560278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.560278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:30.560897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.561675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.562645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:30.562645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.562645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:30.562645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:30.562645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:30.562645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:30.562645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.563322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:30.563322 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.563322 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:30.563322 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.563322 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.563322 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:30.563322 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.563322 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:30.564652 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:30.565107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:30.565107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.565107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:30.565107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.565107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.565107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:30.565107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.565107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:30.566634 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.567148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:30.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:30.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:30.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:30.568337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:30.568744 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:30.569072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.569969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:30.569969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.569969 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:30.569969 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.569969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.569969 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:30.569969 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.569969 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.571808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:30.572326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:35.321424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:35.322853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:35.361219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:35.361594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.362513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:35.362513 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.362513 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:35.362513 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.362513 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.362513 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:35.362513 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.362513 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.363755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.364861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:35.500733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:35.501821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:35.502156 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:35.502399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.503253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:35.503253 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.503253 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:35.503253 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.503253 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.503253 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:35.503253 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.503253 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.504891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.505715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:35.556412 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:35.556842 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:35.557485 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:35.557928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.558362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:35.558362 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.558362 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:35.558362 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.558362 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.558362 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:35.558362 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.558362 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:35.559140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.560404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:35.561966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:35.562851 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.563537 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:35.564738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:35.565264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:35.565264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.565264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:35.565264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.565264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.565264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:35.565264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.565264 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:35.566563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.567076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:35.567598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.567598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:35.567598 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:35.568080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:35.568490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:35.568853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:35.569122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.569858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:35.569858 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.569858 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:35.569858 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.569858 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.569858 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:35.569858 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.569858 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.571069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:35.571770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:40.322145 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:40.323952 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:40.357444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:40.357810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.358658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:40.358658 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.358658 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:40.358658 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.358658 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.358658 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:40.358658 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.358658 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.359548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.360575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:40.501262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:40.502076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:40.502279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:40.502398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.502912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:40.502912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.502912 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:40.502912 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.502912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.502912 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:40.502912 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.502912 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.503728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.504274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:40.556879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:40.557267 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:40.557647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:40.557932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:40.558615 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.558615 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:40.558615 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.558615 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.558615 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:40.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.558615 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:40.559183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:40.559700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:40.559700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.559700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:40.559700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.559700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.559700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:40.559700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.559700 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:40.559996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.560475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.561085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:40.561085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.561085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:40.562315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:40.562720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:40.563264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:40.563613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.564325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:40.564325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.564325 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:40.564325 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.564325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.564325 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:40.564325 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.564325 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:40.565662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:40.566539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:40.566539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.566539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:40.566539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.566539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.566539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:40.566539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.566539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:40.567324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.568234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:40.569224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.569224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:40.569224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:40.569224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:40.569224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:40.569500 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:40.569500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.570360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:40.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:40.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:40.570360 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.570360 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.572506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:40.573425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:45.322893 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:45.324836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:45.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:45.359996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.360633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:45.360633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.360633 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:45.360633 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.360633 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.360633 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:45.360633 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.360633 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.361640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.362200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:45.501903 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:45.502873 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:45.503242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:45.503484 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.504391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:45.504391 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.504391 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:45.504391 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.504391 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.504391 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:45.504391 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.504391 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.506343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.507175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:45.557594 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:45.558157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:45.558821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:45.559168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.559912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:45.559912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.559912 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:45.559912 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.559912 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.559912 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:45.559912 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.559912 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:45.561159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:45.561545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:45.561545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.561545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:45.561545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.561545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.561545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:45.561545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.561545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:45.562763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.563816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.564964 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:45.566108 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.566108 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:45.566963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.567848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:45.568493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:45.569421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:45.569916 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:45.569916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.570354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:45.570354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.570354 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:45.570354 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.570354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.570354 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:45.570354 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.570354 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.571108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:45.571750 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:50.323622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:50.325262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:50.360282 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:50.360594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.361379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:50.361379 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.361379 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:50.361379 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.361379 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.361379 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:50.361379 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.361379 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.362226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.363224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:50.502533 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:50.503890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:50.504385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:50.504629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.505553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:50.505553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.505553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:50.505553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.505553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.505553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:50.505553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.505553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.507236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.508152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:50.558253 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:50.558755 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:50.559332 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:50.559542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.560152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:50.560152 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.560152 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:50.560152 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.560152 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.560152 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:50.560152 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.560152 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:50.561390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:50.562488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:50.562488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.562488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:50.562488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.562488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.562488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:50.562488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.562488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:50.562898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.563731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.564679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:50.564679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.564679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:50.564679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:50.564679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:50.564679 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:50.564937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.565814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:50.565814 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.565814 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:50.565814 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.565814 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.565814 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:50.565814 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.565814 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:50.567392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:50.567840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:50.567840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.567840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:50.567840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.567840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.567840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:50.567840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.567840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:50.569394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.569921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:50.570534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.570534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:50.570534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:50.570534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:50.570534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:50.570713 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:50.570816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.571375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:50.571375 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.571375 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:50.571375 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.571375 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.571375 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:50.571375 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.571375 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.572459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:50.572960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:55.324300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:55.325955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:55.357281 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:55.357686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.358680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:30:55.358680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.358680 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:55.358680 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.358680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.358680 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:55.358680 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.358680 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.359729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.360829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:55.503317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:55.504271 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:55.504624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:55.504865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.505649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:55.505649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.505649 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:55.505649 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.505649 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.505649 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:55.505649 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.505649 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.506718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.507696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:55.559427 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:55.559849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:30:55.560504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:55.560504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.560994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:55.560994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.560994 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:55.560994 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.560994 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.560994 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:55.560994 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.560994 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:55.562431 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:55.563331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:55.563331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.563331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:55.563331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.563331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.563331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:55.563331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.563331 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:30:55.564251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.564702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.565306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:55.565306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.565306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:55.565306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:55.565306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:55.565437 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:55.565536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.566049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:30:55.566049 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.566049 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:55.566049 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.566049 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.566049 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:55.566049 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.566049 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:30:55.566718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:55.567336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:30:55.567336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.567336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:30:55.567336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.567336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.567336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:30:55.567336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.567336 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:30:55.567644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.568079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:55.568486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.568486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:30:55.570079 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:30:55.570548 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:30:55.570841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:55.571037 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:30:55.571149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.571639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:30:55.571639 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.571639 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:30:55.571639 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.571639 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.571639 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:30:55.571639 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.571639 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.572748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:30:55.573243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:00.324866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:00.327547 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:00.371156 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:00.371346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.371907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:00.371907 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.371907 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:00.371907 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.371907 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.371907 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:00.371907 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.371907 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.373582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.374189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:00.507081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:00.508004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:00.508380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:00.508707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.509722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:00.509722 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.509722 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:00.509722 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.509722 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.509722 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:00.509722 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.509722 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.510544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.511057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:00.559925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.562011 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:00.562394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.563015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:00.563570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.564168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:00.564807 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:00.565246 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:00.565696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:00.565775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.567094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.567690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:00.568070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.568575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:00.569157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.569157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:00.570392 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:00.570692 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:00.571039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:00.571248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:00.571314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.571951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:00.571951 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.571951 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:00.571951 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.571951 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.571951 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:00.571951 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.571951 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.572660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:00.573204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:05.326816 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:05.329384 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:05.374823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:05.375332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.375878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:05.375878 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.375878 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:05.375878 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.375878 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.375878 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:05.375878 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.375878 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.377103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.377693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:05.507639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:05.508537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:05.508782 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:05.508946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.509715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:05.509715 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.509715 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:05.509715 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.509715 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.509715 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:05.509715 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.509715 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.511451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.511996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:05.560496 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:05.560945 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:05.562130 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:05.562387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.563006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:05.563006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.563006 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:05.563006 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.563006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.563006 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:05.563006 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.563006 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:05.563662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:05.564190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:05.564190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.564190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:05.564190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.564190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.564190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:05.564190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.564190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:05.564436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.564886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.565432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:05.565432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.565432 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:05.565432 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:05.566564 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:05.566822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:05.567095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.569670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.570477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:05.570908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.571619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:05.572095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.572095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:05.572095 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:05.572095 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:05.572095 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:05.572457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:05.572525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:05.573222 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.573222 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:05.573222 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.573222 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.573222 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:05.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.574914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:05.575434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:10.327411 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:10.328925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:10.365374 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:10.365621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.366230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:10.366230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.366230 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:10.366230 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.366230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.366230 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:10.366230 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.366230 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.367413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.368076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:10.508297 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:10.509328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:10.509646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:10.510166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.510932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:10.510932 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.510932 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:10.510932 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.510932 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.510932 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:10.510932 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.510932 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.511818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.512619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:10.561174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:10.561570 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:10.562115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:10.562333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.562959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:10.562959 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.562959 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:10.562959 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.562959 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.562959 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:10.562959 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.562959 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:10.564470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:10.565767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:10.565767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.565767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:10.565767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.565767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.565767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:10.565767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.565767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:10.566760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.567703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.568592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:10.568592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.568592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:10.568592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:10.568592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:10.568592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:10.568592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.569000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:10.569000 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.569000 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:10.569000 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.569000 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.569000 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:10.569000 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.569000 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:10.569589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:10.570427 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:10.570881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:10.571385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:10.572425 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:10.572425 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:10.572657 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:10.572826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.573363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:10.573363 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.573363 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:10.573363 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.573363 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.573363 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:10.573363 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.573363 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.574949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:10.575458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:15.328149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:15.330004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:15.365014 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:15.365246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.365822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:15.365822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.365822 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:15.365822 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.365822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.365822 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:15.365822 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.365822 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.366926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.367468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:15.509008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:15.509891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:15.510656 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:15.510656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.513038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:15.513672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.514216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.514216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:15.514216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.514216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:15.561757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:15.562485 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:15.562914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:15.563259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.563771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:15.563771 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.563771 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:15.563771 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.563771 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.563771 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:15.563771 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.563771 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:15.565017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:15.565804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:15.565804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.565804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:15.565804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.565804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.565804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:15.565804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.565804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:15.566605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.567385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.568308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:15.568308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.568308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:15.568308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:15.568556 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:15.568909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:15.569117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.569558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:15.569558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.569558 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:15.569558 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.569558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.569558 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:15.569558 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.569558 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:15.570465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.571040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:15.571567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.572067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:15.572597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.572597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:15.572597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:15.572597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:15.572597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.573749 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.574552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:15.575117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:20.328881 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:20.329874 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:20.360574 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:20.360933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.361172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:20.361172 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.361172 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:20.361172 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.361172 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.361172 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:20.361172 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.361172 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.362486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.363929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:20.509451 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:20.510543 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:20.510731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:20.510879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.511455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:20.511455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.511455 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:20.511455 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.511455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.511455 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:20.511455 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.511455 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.512498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.512958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:20.562420 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:20.562641 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:20.563188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:20.563410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.563935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:20.563935 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.563935 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:20.563935 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.563935 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.563935 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:20.563935 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.563935 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:20.564667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.565560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:20.566326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.567400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:20.568350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:20.568833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:20.569316 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:20.569660 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.570360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:20.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:20.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.570360 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:20.570360 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.570360 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:20.571347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.571755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:20.572476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.572969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:20.573470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.573470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:20.573470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:20.573470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:20.573614 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:20.573920 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:20.574183 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.574816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:20.574816 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.574816 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:20.574816 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.574816 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.574816 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:20.574816 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.574816 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.575968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:20.576460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:25.329513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:25.331388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:25.373535 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:25.373808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.374468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:25.374468 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.374468 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:25.374468 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.374468 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.374468 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:25.374468 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.374468 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.375472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.376153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:25.510146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:25.511367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:25.511632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:25.511871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.512680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:25.512680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.512680 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:25.512680 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.512680 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.512680 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:25.512680 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.512680 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.514472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.514811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:25.562900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:25.563480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:25.564228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:25.564533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.565263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:25.565263 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.565263 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:25.565263 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.565263 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.565263 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:25.565263 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.565263 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:25.566225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:25.566692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:25.566692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.566692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:25.566692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.566692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.566692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:25.566692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.566692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:25.567266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.567612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.567942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:25.567942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.567942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:25.568851 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:25.569171 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:25.569616 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:25.569990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.570788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:25.570788 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.570788 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:25.570788 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.570788 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.570788 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:25.570788 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.570788 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:25.572384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:25.572825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:25.572825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.572825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:25.572825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.572825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.572825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:25.572825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.572825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:25.573670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.574200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:25.574686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.574686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:25.574686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:25.574686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:25.574686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:25.574945 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:25.575260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.575785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:25.575785 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.575785 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:25.575785 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.575785 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.575785 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:25.575785 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.575785 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.576824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:25.577338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:30.352226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:30.352665 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:30.354509 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:31:30.381645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:30.382073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.382948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:30.382948 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.382948 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:30.382948 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.382948 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.382948 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:30.382948 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.382948 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.384146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.385017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:30.510922 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:30.511847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:30.512066 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:30.512234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.513002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:30.513002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.513002 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:30.513002 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.513002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.513002 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:30.513002 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.513002 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.514215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.514940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:30.563582 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:30.564045 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:30.564573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:30.564820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.565430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:30.565430 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.565430 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:30.565430 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.565430 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.565430 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:30.565430 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.565430 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:30.566497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:30.567299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:30.567299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.567299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:30.567299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.567299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.567299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:30.567299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.567299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:30.567799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.568546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.569429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:30.569429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.569429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:30.569429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:30.570216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:30.570689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:30.571324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.572047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:30.572047 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.572047 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:30.572047 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.572047 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.572047 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:30.572047 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.572047 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:30.573222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:30.574035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:30.574035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.574035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:30.574035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.574035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.574035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:30.574035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.574035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:30.574856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.575761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:30.576710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.576710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:30.576710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:30.576710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:30.576710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:30.576710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:30.576710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.577225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:30.577225 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.577225 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:30.577225 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.577225 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.577225 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:30.577225 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.577225 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.579003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:30.579732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:35.352280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:35.354333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:35.408019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:35.408439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.409440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:35.409440 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.409440 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:35.409440 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.409440 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.409440 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:35.409440 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.409440 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.411146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.411849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:35.511650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:35.512934 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:35.513303 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:35.513547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.514496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:35.514496 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.514496 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:35.514496 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.514496 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.514496 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:35.514496 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.514496 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.516242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.516716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:35.564220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:35.564697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:35.565654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:35.565856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.566514 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:35.566514 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.566514 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:35.566514 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.566514 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.566514 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:35.566514 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.566514 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:35.567232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.568067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:35.568797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:35.569715 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:35.570575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:35.570758 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:35.571213 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.572078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:35.572078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.572078 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:35.572078 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.572078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.572078 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:35.572078 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.572078 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:35.572899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:35.573651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:35.573651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.573651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:35.573651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.573651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.573651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:35.573651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.573651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:35.573937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.574582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:35.575376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.575376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:35.575481 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:35.575995 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:35.576521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:35.576892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:35.577172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.578051 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:35.578051 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.578051 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:35.578051 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.578051 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.578051 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:35.578051 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.578051 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:35.580398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:40.352940 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:40.354624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:40.413196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:40.413371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.413954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:40.413954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.413954 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:40.413954 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.413954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.413954 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:40.413954 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.413954 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.414745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.415232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:40.512733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:40.513905 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:40.514123 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:40.514222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.515425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:40.515425 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.515425 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:40.515425 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.515425 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.515425 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:40.515425 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.515425 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.517548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.518547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:40.564934 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:40.565522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:40.566242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:40.566565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.567416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:40.567416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.567416 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:40.567416 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.567416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.567416 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:40.567416 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.567416 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:40.568299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:40.569005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:40.569005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.569005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:40.569005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.569005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.569005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:40.569005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.569005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:40.569253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.569699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.570210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:40.570210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.570210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:40.570210 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:40.570699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:40.571220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:40.571589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.572346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:40.572346 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.572346 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:40.572346 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.572346 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.572346 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:40.572346 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.572346 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:40.573595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:40.574049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:40.574049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.574049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:40.574049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.574049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.574049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:40.574049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.574049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:40.575228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.575695 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:40.576229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.576229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:40.576229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:40.576229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:40.576630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:40.577069 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:40.577274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.577991 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:40.577991 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.577991 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:40.577991 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.577991 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.577991 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:40.577991 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.577991 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.579090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:40.579570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:45.353708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:45.355381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:45.393007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:45.393401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.394309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:45.394309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.394309 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:45.394309 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.394309 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.394309 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:45.394309 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.394309 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.395670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.396758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:45.513317 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:45.514503 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:45.514843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:45.515112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.515968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:45.515968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.515968 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:45.515968 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.515968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.515968 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:45.515968 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.515968 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.517219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.518069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:45.565636 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:45.566208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:45.566660 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:45.566862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.567414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:45.567414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.567414 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:45.567414 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.567414 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.567414 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:45.567414 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.567414 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:45.568678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:45.569055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:45.569055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.569055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:45.569055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.569055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.569055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:45.569055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.569055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:45.570408 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.570711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.571074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:45.571074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.571074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:45.571074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:45.571074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:45.571393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:45.571609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.572106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:45.572106 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.572106 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:45.572106 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.572106 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.572106 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:45.572106 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.572106 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:45.572891 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:45.573415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:45.573415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.573415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:45.573415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.573415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.573415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:45.573415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.573415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:45.574046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.574461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:45.574910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.574910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:45.576292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:45.577292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:45.577722 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:45.578081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:45.578300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.579107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:45.579107 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.579107 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:45.579107 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.579107 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.579107 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:45.579107 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.579107 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.580926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:45.581399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:50.354561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:50.356300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:50.386834 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:50.387286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.388121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:50.388121 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.388121 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:50.388121 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.388121 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.388121 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:50.388121 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.388121 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.389005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.389858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:50.514157 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:50.515372 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:50.515707 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:50.515940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.516810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:50.516810 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.516810 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:50.516810 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.516810 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.516810 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:50.516810 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.516810 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.518743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.519478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:50.566387 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:50.566837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:50.567492 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:50.567844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.568726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:50.568726 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.568726 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:50.568726 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.568726 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.568726 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:50.568726 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.568726 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:50.569882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:50.570588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:50.570588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.570588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:50.570588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.570588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.570588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:50.570588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.570588 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:50.571412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.571894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.572448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:50.572448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.572448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:50.572448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:50.572448 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:50.572868 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:50.573284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.574041 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:50.574041 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.574041 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:50.574041 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.574041 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.574041 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:50.574041 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.574041 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:50.574814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:50.575254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:50.575254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.575254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:50.575254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.575254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.575254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:50.575254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.575254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:50.575871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.576343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:50.576822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.576822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:50.576822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:50.577202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:50.577580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:50.577829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:50.578013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.578600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:50.578600 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.578600 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:50.578600 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.578600 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.578600 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:50.578600 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.578600 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.580334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:50.580827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:55.355421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:55.356645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:55.391790 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:55.392081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.392780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:31:55.392780 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.392780 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:55.392780 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.392780 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.392780 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:55.392780 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.392780 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.394392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.394801 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:55.514798 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:55.515956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:55.516236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:55.516420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.517195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:55.517195 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.517195 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:55.517195 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.517195 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.517195 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:55.517195 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.517195 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.518461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.519379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:55.566933 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:55.567521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:31:55.568346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:55.568685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.569580 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:55.569580 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.569580 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:55.569580 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.569580 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.569580 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:55.569580 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.569580 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:55.570741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:55.571626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:55.571626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.571626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:55.571626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.571626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.571626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:55.571626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.571626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:31:55.572060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.572789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.573647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:55.573647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.573647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:55.573647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:55.573647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:55.573647 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:55.573647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.574013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:31:55.574013 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.574013 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:55.574013 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.574013 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.574013 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:55.574013 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.574013 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:31:55.575186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:55.576073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:31:55.576073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.576073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:31:55.576073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.576073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.576073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:31:55.576073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.576073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:31:55.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.578299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:55.578792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.578792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:31:55.578792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:31:55.578792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:31:55.578792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:55.578792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:31:55.578792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.579076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:31:55.579076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.579076 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:31:55.579076 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.579076 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.579076 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:31:55.579076 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.579076 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.580353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:31:55.581014 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:00.356124 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:00.357570 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:00.388950 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:00.389534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.390353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:00.390353 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.390353 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:00.390353 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.390353 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.390353 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:00.390353 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.390353 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.391413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.391939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:00.541637 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:00.543314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:00.543573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:00.543685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.544262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:00.544262 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.544262 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:00.544262 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.544262 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.544262 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:00.544262 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.544262 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.545153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.545558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:00.567567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:00.568121 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:00.568805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:00.569147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.570006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:00.570006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.570006 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:00.570006 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.570006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.570006 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:00.570006 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.570006 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:00.570738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:00.571338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:00.571338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.571338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:00.571338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.571338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.571338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:00.571338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.571338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:00.572925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.573473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.574112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:00.574112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.574112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:00.574112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:00.574112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.576748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.577437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:00.577906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.577906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:00.579048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.579447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:00.579872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.579872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:00.579872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:00.579872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:00.580057 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:00.580363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:00.580543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.581353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:00.581353 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.581353 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:00.581353 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.581353 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.581353 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:00.581353 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.581353 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.582373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:00.582735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:05.356820 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:05.358752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:05.393605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:05.393821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.394403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:05.394403 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.394403 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:05.394403 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.394403 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.394403 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:05.394403 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.394403 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.395527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.396442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:05.542700 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:05.544012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:05.544427 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:05.544550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.545271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:05.545271 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.545271 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:05.545271 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.545271 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.545271 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:05.545271 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.545271 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.571143 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.571503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:05.572173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:05.572173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:05.572173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:05.572173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.572623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:05.572623 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.572623 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:05.572623 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.572623 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.572623 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:05.572623 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.572623 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:05.573085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:05.573459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:05.573459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.573459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:05.573459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.573459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.573459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:05.573459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.573459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:05.573732 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.574082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.574557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:05.574557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.574557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:05.574557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:05.574557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:05.574687 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:05.574839 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.575261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:05.575261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.575261 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:05.575261 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.575261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.575261 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:05.575261 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.575261 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:05.575832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:05.576154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:05.576154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.576154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:05.576154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.576154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.576154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:05.576154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.576154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:05.576512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.576818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:05.577346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.577346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:05.579422 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:05.579769 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:05.580185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:05.580515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:05.580731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.581411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:05.581411 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.581411 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:05.581411 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.581411 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.581411 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:05.581411 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.581411 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.582577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:05.582931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:10.357473 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:10.359402 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:10.388628 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:10.388894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.389678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:10.389678 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.389678 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:10.389678 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.389678 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.389678 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:10.389678 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.389678 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.390861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.391677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:10.543337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:10.544795 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:10.545196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:10.545456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.546568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:10.546568 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.546568 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:10.546568 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.546568 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.546568 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:10.546568 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.546568 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.548858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.549762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:10.571806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:10.572233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:10.572730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:10.572911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.573553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:10.573553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.573553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:10.573553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.573553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.573553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:10.573553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.573553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:10.612905 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.613420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:10.614224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:10.615017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.615337 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:10.615929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.616361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:10.617212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.618029 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.619073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:10.619425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:15.358440 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:15.360062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:15.394470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:15.394653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.395324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:15.395324 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.395324 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:15.395324 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.395324 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.395324 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:15.395324 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.395324 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.396461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.397356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:15.544146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:15.545501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:15.545919 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:15.546266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.547341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:15.547341 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.547341 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:15.547341 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.547341 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.547341 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:15.547341 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.547341 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.549287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.549888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:15.572260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:15.572667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:15.573685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:15.573685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.574453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:15.574453 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.574453 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:15.574453 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.574453 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.574453 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:15.574453 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.574453 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:15.575745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:15.576453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:15.576453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.576453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:15.576453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.576453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.576453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:15.576453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.576453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:15.577318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.578076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.578835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:15.578835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.578835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:15.614008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:15.614628 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:15.615021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:15.658004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.658590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:15.659255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.659860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:15.660257 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:15.660489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:15.660836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:15.661215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:15.661215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:20.359283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:20.361225 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:20.399254 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:20.399944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.400897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:20.400897 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.400897 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:20.400897 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.400897 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.400897 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:20.400897 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.400897 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.401855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.402393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:20.544697 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:20.546696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:20.547278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:20.547556 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.548757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:20.548757 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.548757 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:20.548757 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.548757 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.548757 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:20.548757 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.548757 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.550668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.551150 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:20.572856 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:20.573346 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:20.574511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:20.574989 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.576100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:20.576100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.576100 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:20.576100 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.576100 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.576100 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:20.576100 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.576100 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:20.576945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:20.577519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:20.577519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.577519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:20.577519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.577519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.577519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:20.577519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.577519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:20.577936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.578877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:20.578877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.578877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:20.614596 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:20.614844 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:20.615890 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:20.616418 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.617456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:20.617456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.617456 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:20.617456 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.617456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.617456 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:20.617456 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.617456 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:20.618195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.618630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:20.618927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.619405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:20.619877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.619877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:20.658894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:20.659410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:20.660410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:20.660790 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:20.697856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.699525 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Traceback (most recent call last):
Mar 26 01:32:20.699525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.699525 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:20.699525 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.699525 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.699525 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:20.699525 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.699525 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-012f4cca-bf1e-498a-9816-db8fafcf32c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.700436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:20.700907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:25.359854 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:25.362433 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:25.396539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:25.396907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.397710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:25.397710 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.397710 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:25.397710 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.397710 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.397710 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:25.397710 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.397710 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.398744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.399692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:25.545499 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:25.546648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:25.547194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:25.547719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.548317 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.549517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.550319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:25.573359 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:25.573714 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:25.574204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:25.574670 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.575084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:25.575084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.575084 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:25.575084 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.575084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.575084 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:25.575084 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.575084 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:25.575926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:25.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:25.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:25.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:25.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:25.577092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.578294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.578907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:25.578907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.578907 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:25.615261 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:25.615994 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:25.616909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:25.617358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.618439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:25.618439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.618439 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:25.618439 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.618439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.618439 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:25.618439 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.618439 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:25.619927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.620515 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:25.621924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.622470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:25.622990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.622990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:25.659690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:25.660631 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:25.661264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:25.661730 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:25.662074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.663390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Traceback (most recent call last):
Mar 26 01:32:25.663390 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.663390 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:25.663390 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.663390 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.663390 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:25.663390 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.663390 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-b72426c9-415b-4dd5-9d65-a07b29214592 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.665238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:25.666087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:30.361450 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:30.363019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:30.364828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:32:30.401725 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:30.402088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.402626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:30.402626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.402626 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:30.402626 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.402626 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.402626 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:30.402626 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.402626 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.403565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.404587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:30.546351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:30.547623 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:30.547923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:30.548241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.549163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:30.549163 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.549163 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:30.549163 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.549163 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.549163 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:30.549163 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.549163 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.550821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.551707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:30.573782 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:30.574112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:30.574603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:30.574957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.575769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:30.575769 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.575769 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:30.575769 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.575769 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.575769 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:30.575769 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.575769 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:30.577157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:30.578038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:30.578038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.578038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:30.578038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.578038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.578038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:30.578038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.578038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:30.579296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.580167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.581219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:30.581219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.581219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:30.615685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:30.616763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:30.617544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:30.617959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.618743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:30.618743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.618743 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:30.618743 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.618743 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.618743 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:30.618743 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.618743 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:30.620594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.621319 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:30.622444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.623149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:30.623606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.623606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:30.660477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:30.661183 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:30.661723 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:30.662154 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:30.662393 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.663433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:32:30.663433 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.663433 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:30.663433 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.663433 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.663433 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:30.663433 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.663433 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.664625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:30.665458 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:35.362103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:35.363822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:35.397609 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:35.398181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.399068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:35.399068 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.399068 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:35.399068 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.399068 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.399068 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:35.399068 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.399068 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.399921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.400858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:35.546946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:35.548275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:35.548477 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:35.548609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.549406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:35.549406 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.549406 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:35.549406 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.549406 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.549406 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:35.549406 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.549406 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.551109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.552308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:35.574311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:35.575022 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:35.575727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:35.576039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.576809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:35.576809 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.576809 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:35.576809 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.576809 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.576809 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:35.576809 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.576809 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:35.578636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.580792 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:35.581601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.582392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:35.616437 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:35.616827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:35.617311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:35.617592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.618775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:35.618775 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.618775 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:35.618775 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.618775 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.618775 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:35.618775 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.618775 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:35.619866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.620623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:35.621599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.622119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:35.622632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.622632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:35.660957 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:35.661548 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:35.662305 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:35.662712 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:35.663009 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.664006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:32:35.664006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.664006 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:35.664006 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.664006 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.664006 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:35.664006 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.664006 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.665709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:35.667060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:40.362924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:40.364587 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:40.396914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:40.397401 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.398246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:40.398246 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.398246 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:40.398246 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.398246 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.398246 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:40.398246 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.398246 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.399631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.400417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:40.547827 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:40.549236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:40.549601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:40.549853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.550804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:40.550804 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.550804 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:40.550804 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.550804 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.550804 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:40.550804 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.550804 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.552453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.553295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:40.574875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:40.575514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:40.576172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:40.576509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.578664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:40.578664 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.578664 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:40.578664 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.578664 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.578664 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:40.578664 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.578664 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:40.579475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:40.580205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:40.580205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.580205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:40.580205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.580205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.580205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:40.580205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.580205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:40.580994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.581701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.582541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:40.582541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.582541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:40.616892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:40.617567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:40.618171 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:40.618539 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.619355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:40.619355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.619355 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:40.619355 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.619355 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.619355 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:40.619355 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.619355 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:40.620659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.621496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:40.622433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.623303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:40.624219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.624219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:40.661540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:40.662121 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:40.662532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:40.662813 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:40.663005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.663450 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:32:40.663450 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.663450 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:40.663450 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.663450 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.663450 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:40.663450 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.663450 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.664499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:40.665365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:45.363671 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:45.367014 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:45.403182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:45.403880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.404596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:45.404596 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.404596 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:45.404596 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.404596 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.404596 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:45.404596 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.404596 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.405721 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.406493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:45.548742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:45.549867 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:45.550691 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:45.550691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.551499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:45.551499 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.551499 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:45.551499 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.551499 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.551499 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:45.551499 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.551499 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.553249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.554161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:45.575595 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:45.576052 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:45.576517 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:45.576830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.577600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:45.577600 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.577600 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:45.577600 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.577600 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.577600 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:45.577600 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.577600 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:45.578783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:45.579278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:45.579278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.579278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:45.579278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.579278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.579278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:45.579278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.579278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:45.580396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.581279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.582184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:45.582184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.582184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:45.617545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:45.618110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:45.618691 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:45.619154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.619931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:45.619931 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.619931 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:45.619931 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.619931 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.619931 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:45.619931 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.619931 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:45.620834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.621737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:45.622160 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.622782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:45.623293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.623293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:45.662114 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:45.662659 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:45.663203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:45.663456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:45.663704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.664471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:32:45.664471 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.664471 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:45.664471 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.664471 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.664471 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:45.664471 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.664471 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.666072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:45.667039 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:50.365550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:50.367435 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:50.404889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:50.405410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.406439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:50.406439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.406439 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:50.406439 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.406439 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.406439 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:50.406439 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.406439 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.407628 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.408442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:50.549594 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:50.550763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:50.551840 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:50.551952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.552969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:50.552969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.552969 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:50.552969 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.552969 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.552969 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:50.552969 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.552969 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.554165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.555249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:50.576136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:50.576575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:50.577182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:50.577467 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.578255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:50.578255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.578255 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:50.578255 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.578255 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.578255 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:50.578255 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.578255 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:50.579552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:50.580587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:50.580587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.580587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:50.580587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.580587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.580587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:50.580587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.580587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:50.581414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.582147 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.582930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:50.582930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.582930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:50.618097 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:50.618749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:50.619389 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:50.619760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.620528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:50.620528 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.620528 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:50.620528 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.620528 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.620528 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:50.620528 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.620528 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:50.621953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.622607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:50.623838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.624571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:50.625435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.625435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:50.662833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:50.663469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:50.664062 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:50.664410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:50.664626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.665388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:32:50.665388 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.665388 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:50.665388 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.665388 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.665388 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:50.665388 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.665388 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.666998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:50.667816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:55.366325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:55.367493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:55.410116 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:55.410664 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.411767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:32:55.411767 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.411767 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:55.411767 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.411767 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.411767 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:55.411767 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.411767 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.413053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.414012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:55.550237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:55.551110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:55.551266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:55.551426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.551935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:55.551935 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.551935 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:55.551935 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.551935 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.551935 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:55.551935 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.551935 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.552875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.553754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:55.576553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:55.576889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:32:55.577351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:55.577622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.578347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:32:55.578347 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.578347 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:55.578347 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.578347 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.578347 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:55.578347 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.578347 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:55.578962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.579873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:55.580756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.581642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:55.618998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:55.619628 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:55.620202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:55.620538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.621561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:32:55.621561 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.621561 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:55.621561 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.621561 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.621561 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:55.621561 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.621561 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:32:55.623235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.624225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:32:55.625066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.626139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:55.627284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.627284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:32:55.663506 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:32:55.664226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:32:55.664779 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:55.665167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:32:55.665380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.666224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:32:55.666224 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.666224 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:32:55.666224 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.666224 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.666224 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:32:55.666224 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.666224 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.667915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:32:55.668854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:00.366944 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:00.368104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:00.399593 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:00.400192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.401107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:00.401107 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.401107 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:00.401107 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.401107 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.401107 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:00.401107 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.401107 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.402282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.402879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:00.550869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:00.551943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:00.552369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:00.552775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.554024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:00.554024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.554024 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:00.554024 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.554024 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.554024 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:00.554024 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.554024 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.556234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.557082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:00.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:00.577502 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:00.578179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:00.578489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.579270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:00.579270 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.579270 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:00.579270 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.579270 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.579270 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:00.579270 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.579270 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:00.580654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:00.581175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:00.581175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.581175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:00.581175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.581175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.581175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:00.581175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.581175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:00.582364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.582803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.583335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:00.583335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.583335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:00.619511 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:00.620041 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:00.620726 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:00.621095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.621755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:00.621755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.621755 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:00.621755 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.621755 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.621755 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:00.621755 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.621755 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:00.622533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.623270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:00.623724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.624782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:00.625656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.625656 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:00.664201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:00.664898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:00.665456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:00.665846 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:00.666057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.666811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:00.666811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.666811 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:00.666811 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.666811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.666811 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:00.666811 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.666811 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.669093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:00.669949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:05.367558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:05.369621 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:05.402101 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:05.402101 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.402560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:05.402560 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.402560 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:05.402560 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.402560 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.402560 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:05.402560 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.402560 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.403398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.404286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:05.551389 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:05.552391 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:05.552550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:05.552714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.553346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:05.553346 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.553346 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:05.553346 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.553346 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.553346 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:05.553346 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.553346 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.554922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.555782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:05.577570 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:05.578240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:05.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:05.579070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.579502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:05.579502 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.579502 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:05.579502 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.579502 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.579502 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:05.579502 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.579502 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:05.580755 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:05.581558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:05.581558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.581558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:05.581558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.581558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.581558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:05.581558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.581558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:05.582472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.583214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.584043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:05.584043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.584043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:05.620154 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:05.620794 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:05.621399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:05.621613 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.622356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:05.622356 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.622356 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:05.622356 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.622356 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.622356 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:05.622356 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.622356 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:05.623587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.624028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:05.625215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.625875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:05.626715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.626715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:05.664900 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:05.665345 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:05.665753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:05.666054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:05.666226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.666838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:05.666838 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.666838 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:05.666838 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.666838 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.666838 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:05.666838 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.666838 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.668435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:05.669169 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:10.368391 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:10.369953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:10.401839 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:10.402096 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.402811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:10.402811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.402811 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:10.402811 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.402811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.402811 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:10.402811 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.402811 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.403665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.404536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:10.552082 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:10.553484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:10.553789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:10.554082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.554893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:10.554893 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.554893 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:10.554893 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.554893 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.554893 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:10.554893 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.554893 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.556019 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.556849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:10.578130 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:10.578721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:10.579435 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:10.579946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.580886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:10.580886 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.580886 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:10.580886 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.580886 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.580886 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:10.580886 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.580886 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:10.582210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:10.583196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:10.583196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.583196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:10.583196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.583196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.583196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:10.583196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.583196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:10.583736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.584465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.585352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:10.585352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.585352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:10.620865 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:10.621472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:10.622217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:10.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.623519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:10.623519 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.623519 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:10.623519 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.623519 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.623519 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:10.623519 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.623519 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:10.624965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.626032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:10.627260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.628140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:10.628946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.628946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:10.665553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:10.666234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:10.666775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:10.667200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:10.667435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.668697 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:10.668697 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.668697 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:10.668697 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.668697 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.668697 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:10.668697 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.668697 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.669928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:10.670943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:15.368915 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:15.370172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:15.401633 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:15.401921 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.402606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:15.402606 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.402606 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:15.402606 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.402606 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.402606 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:15.402606 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.402606 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.403799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.404453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:15.552939 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:15.554344 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:15.554344 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:15.554552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.555013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:15.555013 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.555013 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:15.555013 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.555013 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.555013 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:15.555013 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.555013 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.556200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.557197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:15.578619 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:15.579105 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:15.579762 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:15.580025 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.580619 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:15.580619 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.580619 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:15.580619 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.580619 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.580619 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:15.580619 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.580619 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:15.581770 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:15.582795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:15.582795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.582795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:15.582795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.582795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.582795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:15.582795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.582795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:15.583263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.583966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.584795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:15.584795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.584795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:15.622133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:15.622133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:15.622927 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:15.623395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.624272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:15.624272 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.624272 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:15.624272 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.624272 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.624272 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:15.624272 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.624272 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:15.625710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.626295 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:15.627479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.628196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:15.628688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.628688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:15.666112 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:15.666857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:15.667530 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:15.667925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:15.668383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.669113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:15.669113 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.669113 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:15.669113 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.669113 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.669113 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:15.669113 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.669113 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:15.672061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:20.369491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:20.371104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:20.399705 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:20.400162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.401182 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:20.401182 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.401182 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:20.401182 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.401182 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.401182 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:20.401182 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.401182 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.402057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.402841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:20.553749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:20.554731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:20.554902 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:20.555078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.555621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:20.555621 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.555621 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:20.555621 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.555621 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.555621 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:20.555621 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.555621 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.556859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.557818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:20.579429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:20.579929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:20.580522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:20.580841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.581582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:20.581582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.581582 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:20.581582 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.581582 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.581582 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:20.581582 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.581582 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:20.582683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:20.583451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:20.583451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.583451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:20.583451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.583451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.583451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:20.583451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.583451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:20.584824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.585368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.585915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:20.585915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.585915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:20.623342 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:20.623342 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:20.623593 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:20.624114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.625027 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:20.625027 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.625027 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:20.625027 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.625027 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.625027 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:20.625027 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.625027 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:20.626346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.627097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:20.627995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.628931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:20.629795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.629795 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:20.666728 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:20.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:20.669063 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:20.669434 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:20.669606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.670782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:20.670782 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.670782 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:20.670782 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.670782 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.670782 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:20.670782 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.670782 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.672728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:20.673207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:24.808410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:24.865382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 26 01:33:24.865540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105681) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}}
Mar 26 01:33:24.865854 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Fixing resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:33:24.902377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, resource={'id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'name': 'private', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['02d96a4b-4c93-4b95-ad0f-f11d085f0699', '0503b55a-1b69-49b5-b418-16ba37be90e9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T00:58:10Z', 'updated_at': '2026-03-26T00:58:13Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4449}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.902799 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.903387 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6fe31530-ff9f-476b-aea1-4804059b76cd, external_ids_update=None, columns={'tag': 4449}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.903572 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.904524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Traceback (most recent call last):
Mar 26 01:33:24.904524 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:24.904524 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:24.904524 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:24.904524 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:24.904524 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:24.904524 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.904524 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Failed to fix resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:24.905846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:33:24.906473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:24.907145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:33:24.907145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.907145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:33:24.907145 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Fixing resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:33:24.935791 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-db438906-c74f-4dc8-883c-6e56962e551b, resource={'id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'name': 'public', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['1c3e4583-6e63-4953-8050-3341662a5ec6', '23682989-62bf-4e07-a74f-0cb01077d5ac'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-26T00:58:16Z', 'updated_at': '2026-03-26T00:58:25Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.936181 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-db438906-c74f-4dc8-883c-6e56962e551b, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.936773 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-200618ec-0540-41b0-9429-dc7dfc74abb0, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.937134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.938007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Traceback (most recent call last):
Mar 26 01:33:24.938007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:24.938007 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:24.938007 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:24.938007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:24.938007 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:24.938007 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.938007 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Failed to fix resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:24.938959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:33:24.939526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:24.940266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:33:24.940266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.940266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:33:24.940266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Fixing resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:33:24.962806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, resource={'id': 'f7e819e4-7101-412b-b047-baa5e16b2a52', 'name': 'admin_net', 'tenant_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'project_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f2e900ee-4d88-4151-8e50-661745950632'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:14Z', 'updated_at': '2026-03-26T01:02:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4803}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.963159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.963663 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d43d91f1-f24a-411f-bae7-af85048f95b9, external_ids_update=None, columns={'tag': 4803}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.963861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.964620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Traceback (most recent call last):
Mar 26 01:33:24.964620 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:24.964620 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:24.964620 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:24.964620 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:24.964620 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:24.964620 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.964620 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Failed to fix resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:24.965435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:33:24.966087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:24.966866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:33:24.966866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.966866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:33:24.966866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Fixing resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:33:24.989017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, resource={'id': 'f481c2f9-5052-46d5-821a-6c8712ec70ec', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['329ea1d3-f656-45b1-bf23-1d927d4578ea'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:50Z', 'updated_at': '2026-03-26T01:02:51Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19903}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.989400 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.990397 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3aa5be7a-5c67-4f1a-bdf2-7bf6912e871d, external_ids_update=None, columns={'tag': 19903}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:24.990794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.991312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Traceback (most recent call last):
Mar 26 01:33:24.991312 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:24.991312 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:24.991312 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:24.991312 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:24.991312 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:24.991312 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.991312 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Failed to fix resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:24.992190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:33:24.992777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:24.993464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:33:24.993464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:24.993464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:33:24.993464 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Fixing resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:33:25.057187 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, resource={'id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'enable_snat': True}, 'gw_port_id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-ryt6n', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'ip_address': '172.24.5.142', 'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac'}, {'ip_address': '2001:db8::16a', 'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 3, 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8'}, resource_type=routers, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.057556 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.057792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.058221 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.058221 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.058747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Traceback (most recent call last):
Mar 26 01:33:25.058747 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.058747 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:25.058747 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.058747 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.058747 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:25.058747 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.058747 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:25.059191 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Unable to update router e4efd798-3c25-4ea9-bd45-fa7a05aeaef9. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Failed to fix resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 26 01:33:25.060424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:33:25.060811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:33:25.061178 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Fixing resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:33:25.154613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, resource={'id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'name': '', 'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'project_id': '', 'mac_address': 'fa:16:3e:4e:d4:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160849', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.154880 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::16a/64', '172.24.5.142/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.155393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=14a0aa42-40aa-4508-9af3-cffdffef9818, lrouter_port=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.155744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.156571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Traceback (most recent call last):
Mar 26 01:33:25.156571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.156571 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:25.156571 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.156571 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.156571 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:25.156571 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.156571 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Failed to fix resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:25.157453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.158073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.158639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:33:25.158639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.158639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:33:25.158639 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0a42c168-1cd0-410d-95b9-60cf8aff80e9 None None] Maintenance task: Synchronization completed (took 0.29 seconds)
Mar 26 01:33:25.370264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:25.371829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:25.405300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.405747 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.406683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:25.406683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.406683 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:25.406683 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.406683 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.406683 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:25.406683 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.406683 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.407909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.408776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:25.554340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:25.555580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.555799 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.555936 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.556669 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:25.556669 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.556669 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:25.556669 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.556669 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.556669 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:25.556669 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.556669 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.558506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.559210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:25.579948 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:25.580384 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:25.580805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.581109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.581968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:25.581968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.581968 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:25.581968 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.581968 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.581968 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:25.581968 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.581968 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:25.582908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.583758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:25.583758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.583758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:25.583758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.583758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.583758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:25.583758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.583758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:25.584415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.584945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.585506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:25.585506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.585506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:25.622490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:25.623379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:25.624155 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.624567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.625417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:25.625417 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.625417 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:25.625417 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.625417 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.625417 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:25.625417 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.625417 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:25.626339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.626749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:25.627421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.627913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:25.628387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.628387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:25.667635 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:25.668530 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:25.669073 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.669444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:25.669667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.670415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:25.670415 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.670415 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:25.670415 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.670415 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.670415 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:25.670415 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.670415 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.671568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:25.672133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:30.371071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:30.372823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:30.374544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=434 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:33:30.408558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:30.408967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.409931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:30.409931 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.409931 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:30.409931 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.409931 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.409931 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:30.409931 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.409931 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.411228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.411777 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:30.555164 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:30.556308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:30.556637 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:30.556884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.557773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:30.557773 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.557773 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:30.557773 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.557773 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.557773 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:30.557773 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.557773 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.558945 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.559798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:30.580576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:30.580924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:30.581380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:30.581730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.582857 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:30.582857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.582857 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:30.582857 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.582857 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.582857 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:30.582857 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.582857 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:30.583662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:30.584496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:30.584496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.584496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:30.584496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.584496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.584496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:30.584496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.584496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:30.584893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.585612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.586518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:30.586518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.586518 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:30.624375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:30.624375 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:30.624879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:30.625492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.625967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:30.625967 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.625967 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:30.625967 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.625967 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.625967 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:30.625967 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.625967 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:30.626944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.627658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:30.628069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.628949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:30.629856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.629856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:30.668335 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:30.668699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:30.669122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:30.669518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:30.669731 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.670436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:30.670436 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.670436 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:30.670436 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.670436 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.670436 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:30.670436 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.670436 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.671468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:30.672347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:35.371837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:35.373635 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:35.417612 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:35.418404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.419543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:35.419543 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.419543 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:35.419543 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.419543 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.419543 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:35.419543 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.419543 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.421312 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.421862 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:35.555797 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:35.557268 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:35.557577 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:35.557865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.558811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:35.558811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.558811 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:35.558811 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.558811 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.558811 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:35.558811 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.558811 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.560681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.561241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:35.580969 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:35.581791 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:35.582260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:35.582593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.583392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:35.583392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.583392 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:35.583392 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.583392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.583392 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:35.583392 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.583392 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:35.584818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:35.585379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:35.585379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.585379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:35.585379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.585379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.585379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:35.585379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.585379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:35.586505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.586993 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.587546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:35.587546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.587546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:35.624470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:35.625216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:35.625997 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:35.626451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.627392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:35.627392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.627392 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:35.627392 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.627392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.627392 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:35.627392 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.627392 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:35.628719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.629466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:35.630380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.631149 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:35.631953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.631953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:35.669265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:35.669776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:35.670318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:35.670727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:35.670947 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.671745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:35.671745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.671745 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:35.671745 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.671745 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.671745 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:35.671745 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.671745 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.673342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:35.674228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:40.372921 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:40.374623 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:40.404522 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:40.404997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.406189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:40.406189 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.406189 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:40.406189 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.406189 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.406189 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:40.406189 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.406189 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.407008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.407886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:40.556560 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:40.558219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:40.558560 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:40.558804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.559908 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:40.559908 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.559908 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:40.559908 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.559908 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.559908 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:40.559908 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.559908 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.561692 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.562917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:40.581534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:40.581877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:40.582527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:40.582946 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.583822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:40.583822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.583822 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:40.583822 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.583822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.583822 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:40.583822 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.583822 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:40.585080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:40.586257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:40.586257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.586257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:40.586257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.586257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.586257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:40.586257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.586257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:40.586794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.587884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.589134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:40.589134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.589134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:40.625151 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:40.625646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:40.626382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:40.626785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.627608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:40.627608 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.627608 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:40.627608 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.627608 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.627608 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:40.627608 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.627608 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:40.628844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.629813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:40.630751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.631766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:40.632870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.632870 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:40.669763 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:40.670203 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:40.670864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:40.671196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:40.671347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.672005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:40.672005 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.672005 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:40.672005 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.672005 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.672005 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:40.672005 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.672005 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.673000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:40.674171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:45.373626 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:45.374586 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:45.405880 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:45.406202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.406841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:45.406841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.406841 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:45.406841 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.406841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.406841 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:45.406841 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.406841 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.407685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.408217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:45.557092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:45.558052 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:45.558914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:45.558914 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.559124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:45.559124 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.559124 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:45.559124 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.559124 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.559124 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:45.559124 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.559124 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.560188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.560674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:45.581886 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:45.582095 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:45.582829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:45.583026 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.583694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:45.583694 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.583694 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:45.583694 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.583694 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.583694 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:45.583694 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.583694 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:45.584448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:45.585017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:45.585017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.585017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:45.585017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.585017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.585017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:45.585017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.585017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:45.585344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.585790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.586355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:45.586355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.586355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:45.626514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:45.627959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:45.627959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:45.627959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.628536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:45.628536 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.628536 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:45.628536 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.628536 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.628536 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:45.628536 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.628536 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:45.629769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.630567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:45.631121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.631609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:45.632125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.632125 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:45.670193 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:45.670676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:45.671288 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:45.671602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:45.671727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.672601 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:45.672601 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.672601 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:45.672601 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.672601 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.672601 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:45.672601 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.672601 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.674309 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:45.675157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:50.374236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:50.375539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:50.406006 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:50.406346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.407267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:50.407267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.407267 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:50.407267 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.407267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.407267 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:50.407267 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.407267 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.408767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.409687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:50.557699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:50.558955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:50.559340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:50.559616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.560500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:50.560500 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.560500 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:50.560500 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.560500 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.560500 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:50.560500 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.560500 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.561698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.562546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:50.582269 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:50.582623 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:50.583172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:50.583483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.584234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:50.584234 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.584234 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:50.584234 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.584234 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.584234 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:50.584234 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.584234 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:50.585263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:50.586146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:50.586146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.586146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:50.586146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.586146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.586146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:50.586146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.586146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:50.586529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.587281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.588134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:50.588134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.588134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:50.627078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:50.627571 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:50.628154 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:50.628543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.629416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:50.629416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.629416 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:50.629416 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.629416 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.629416 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:50.629416 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.629416 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:50.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.631412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:50.632492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.633691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:50.634575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.634575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:50.670729 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:50.671306 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:50.671837 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:50.672224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:50.672440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.673362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:50.673362 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.673362 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:50.673362 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.673362 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.673362 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:50.673362 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.673362 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.675188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:50.676022 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:55.374929 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:55.376587 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:55.407674 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:55.408066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.408760 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:33:55.408760 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.408760 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:55.408760 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.408760 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.408760 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:55.408760 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.408760 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.409537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.410791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:55.558575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:55.559704 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:55.559959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:55.560083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.560822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:55.560822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.560822 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:55.560822 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.560822 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.560822 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:55.560822 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.560822 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.562690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.566168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:55.582745 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:55.583292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:33:55.584072 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:55.584072 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.584809 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:33:55.584809 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.584809 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:55.584809 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.584809 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.584809 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:55.584809 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.584809 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:55.585948 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:55.587324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:55.587324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.587324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:55.587324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.587324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.587324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:55.587324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.587324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:33:55.587899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.589040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.590348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:55.590348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.590348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:55.627782 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:55.628242 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:55.628756 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:55.629127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.629909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:33:55.629909 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.629909 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:55.629909 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.629909 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.629909 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:55.629909 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.629909 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:33:55.631399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.632197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:33:55.633156 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.634021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:55.634956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.634956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:33:55.671231 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:33:55.671946 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:33:55.672449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:55.672800 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:33:55.673050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.673772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:33:55.673772 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.673772 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:33:55.673772 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.673772 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.673772 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:33:55.673772 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.673772 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.675560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:33:55.676629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:00.375661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:00.377451 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:00.415106 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:00.415333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.415949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:00.415949 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.415949 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:00.415949 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.415949 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.415949 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:00.415949 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.415949 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.417392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.418346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:00.559234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:00.560590 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:00.560930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:00.561208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.562007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:00.562007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.562007 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:00.562007 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.562007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.562007 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:00.562007 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.562007 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.564779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:00.583364 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:00.583654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:00.584082 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:00.584390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.585007 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:00.585007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.585007 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:00.585007 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.585007 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.585007 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:00.585007 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.585007 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:00.586122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:00.586622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:00.586622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.586622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:00.586622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.586622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.586622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:00.586622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.586622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:00.587603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.588202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.588743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:00.588743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.588743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:00.628475 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:00.629142 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:00.630376 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:00.630376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.630734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:00.630734 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.630734 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:00.630734 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.630734 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.630734 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:00.630734 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.630734 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:00.632052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.632682 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:00.633701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.634453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:00.635310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.635310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:00.671791 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:00.672531 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:00.673110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:00.673491 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:00.673740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.674526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:00.674526 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.674526 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:00.674526 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.674526 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.674526 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:00.674526 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.674526 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.676287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:00.677292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:05.376453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:05.378179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:05.413394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:05.413591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.413956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:05.413956 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.413956 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:05.413956 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.413956 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.413956 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:05.413956 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.413956 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.414706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.415608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:05.559790 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:05.561123 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:05.561398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:05.561559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.562371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:05.562371 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.562371 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:05.562371 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.562371 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.562371 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:05.562371 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.562371 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.564369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.565138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:05.583864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:05.584217 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:05.584520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:05.584718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.585161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:05.585161 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.585161 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:05.585161 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.585161 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.585161 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:05.585161 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.585161 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:05.585888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:05.586699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:05.586699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.586699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:05.586699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.586699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.586699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:05.586699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.586699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:05.587120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.587811 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.588627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:05.588627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.588627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:05.629078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:05.629682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:05.630394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:05.630710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.631283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:05.631283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.631283 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:05.631283 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.631283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.631283 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:05.631283 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.631283 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:05.632274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.632970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:05.633452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.634276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:05.635193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.635193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:05.672298 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:05.672971 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:05.673513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:05.673847 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:05.674151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.674963 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:05.674963 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.674963 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:05.674963 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.674963 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.674963 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:05.674963 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.674963 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.676653 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:05.677598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:10.377311 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:10.378822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:10.413711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:10.413869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.414274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:10.414274 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.414274 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:10.414274 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.414274 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.414274 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:10.414274 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.414274 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.414994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.415956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:10.560646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:10.562019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:10.562264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:10.562478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.563557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:10.563557 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.563557 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:10.563557 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.563557 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.563557 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:10.563557 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.563557 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.564494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.565207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:10.584476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:10.584894 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:10.585544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:10.585797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.586516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:10.586516 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.586516 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:10.586516 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.586516 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.586516 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:10.586516 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.586516 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:10.587968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:10.588314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:10.588314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.588314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:10.588314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.588314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.588314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:10.588314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.588314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:10.589716 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.590070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.590428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:10.590428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.590428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:10.629768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:10.630239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:10.630601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:10.630847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.631426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:10.631426 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.631426 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:10.631426 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.631426 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.631426 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:10.631426 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.631426 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:10.632644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.632964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:10.634260 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.634584 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:10.634924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.634924 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:10.672875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:10.673568 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:10.673962 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:10.674223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:10.674376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.674930 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:10.674930 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.674930 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:10.674930 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.674930 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.674930 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:10.674930 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.674930 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.676769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:10.677152 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:15.377972 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:15.379232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:15.409573 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:15.409971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.410995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:15.410995 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.410995 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:15.410995 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.410995 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.410995 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:15.410995 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.410995 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.412036 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.412910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:15.561352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:15.562710 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:15.562940 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:15.563059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.563854 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:15.563854 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.563854 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:15.563854 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.563854 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.563854 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:15.563854 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.563854 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.565650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:15.585065 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:15.585490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:15.586073 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:15.586352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.586999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:15.586999 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.586999 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:15.586999 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.586999 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.586999 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:15.586999 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.586999 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:15.588188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:15.589050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:15.589050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.589050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:15.589050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.589050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.589050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:15.589050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.589050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:15.589686 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.590509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.591410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:15.591410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.591410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:15.630530 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:15.631100 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:15.631716 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:15.633131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.633960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:15.633960 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.633960 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:15.633960 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.633960 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.633960 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:15.633960 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.633960 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:15.635380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.636024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:15.637712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.638448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:15.639238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.639238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:15.673493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:15.674226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:15.674783 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:15.675211 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:15.675447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.676261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:15.676261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.676261 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:15.676261 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.676261 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.676261 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:15.676261 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.676261 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.677969 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:15.678498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:20.378702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:20.380229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:20.415966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:20.416395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.417251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:20.417251 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.417251 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:20.417251 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.417251 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.417251 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:20.417251 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.417251 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.418219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.419097 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:20.562041 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:20.563479 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:20.563801 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:20.564093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.564933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:20.564933 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.564933 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:20.564933 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.564933 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.564933 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:20.564933 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.564933 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.567135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.567873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:20.585524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:20.585898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:20.586445 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:20.586733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.587435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:20.587435 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.587435 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:20.587435 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.587435 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.587435 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:20.587435 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.587435 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:20.588621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:20.589438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:20.589438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.589438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:20.589438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.589438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.589438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:20.589438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.589438 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:20.590141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.590899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.591800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:20.591800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.591800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:20.631077 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:20.631597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:20.632289 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:20.632687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.633485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:20.633485 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.633485 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:20.633485 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.633485 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.633485 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:20.633485 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.633485 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:20.635032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.635673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:20.636684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.637389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:20.638177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.638177 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:20.674110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:20.674671 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:20.675077 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:20.675406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:20.675626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.676381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:20.676381 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.676381 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:20.676381 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.676381 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.676381 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:20.676381 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.676381 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.677887 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:20.678743 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:25.379439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:25.381025 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:25.417012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:25.417366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.418278 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:25.418278 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.418278 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:25.418278 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.418278 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.418278 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:25.418278 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.418278 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.419586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.420899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:25.562691 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:25.563720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:25.563934 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:25.564110 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.564752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:25.564752 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.564752 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:25.564752 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.564752 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.564752 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:25.564752 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.564752 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.565937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.566849 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:25.586035 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:25.586650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:25.587050 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:25.587273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.587847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:25.587847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.587847 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:25.587847 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.587847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.587847 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:25.587847 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.587847 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:25.589340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:25.590508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:25.590508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.590508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:25.590508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.590508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.590508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:25.590508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.590508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:25.591684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.592507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.593504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:25.593504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.593504 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:25.631828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:25.632557 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:25.633180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:25.633455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.634114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:25.634114 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.634114 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:25.634114 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.634114 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.634114 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:25.634114 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.634114 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:25.635361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.636083 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:25.637005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.637775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:25.638762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.638762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:25.674650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:25.675086 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:25.675331 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:25.675486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:25.675614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.675996 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:25.675996 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.675996 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:25.675996 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.675996 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.675996 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:25.675996 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.675996 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.676733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:25.677775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:30.380096 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:30.381022 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:30.382301 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:34:30.414027 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:30.414286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.415258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:30.415258 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.415258 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:30.415258 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.415258 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.415258 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:30.415258 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.415258 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.416017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.416925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:30.563419 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:30.565731 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:30.566152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:30.566507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.567509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:30.567509 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.567509 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:30.567509 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.567509 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.567509 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:30.567509 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.567509 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.568589 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.569229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:30.586709 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:30.587327 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:30.588019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:30.588474 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.589352 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:30.589352 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.589352 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:30.589352 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.589352 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.589352 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:30.589352 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.589352 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:30.590557 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:30.591453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:30.591453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.591453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:30.591453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.591453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.591453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:30.591453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.591453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:30.592612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.593457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.594330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:30.594330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.594330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:30.632616 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:30.633103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:30.633659 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:30.633937 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.634867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:30.634867 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.634867 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:30.634867 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.634867 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.634867 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:30.634867 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.634867 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:30.635752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:30.637057 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:30.637633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.638373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:30.675272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:30.675775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:30.676202 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:30.676466 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:30.676646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.677212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:30.677212 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.677212 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:30.677212 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.677212 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.677212 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:30.677212 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.677212 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.678060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:30.678529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:35.380850 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:35.382733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:35.416956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:35.417274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.418133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:35.418133 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.418133 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:35.418133 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.418133 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.418133 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:35.418133 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.418133 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.419118 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.420069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:35.564808 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:35.565831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:35.566193 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:35.566424 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.567200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:35.567200 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.567200 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:35.567200 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.567200 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.567200 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:35.567200 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.567200 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.569094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.569861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:35.587275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:35.587687 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:35.588412 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:35.588640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.589339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:35.589339 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.589339 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:35.589339 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.589339 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.589339 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:35.589339 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.589339 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:35.590668 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:35.591600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:35.591600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.591600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:35.591600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.591600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.591600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:35.591600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.591600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:35.592266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.593122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.594034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:35.594034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.594034 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:35.633195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:35.633810 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:35.634362 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:35.634696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.635559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:35.635559 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.635559 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:35.635559 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.635559 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.635559 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:35.635559 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.635559 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:35.637105 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.637771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:35.638717 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.639472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:35.640242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.640242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:35.675876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:35.676413 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:35.676786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:35.677170 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:35.677396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.678151 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:35.678151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.678151 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:35.678151 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.678151 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.678151 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:35.678151 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.678151 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.679794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:35.680607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:40.381717 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:40.383271 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:40.420936 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:40.421382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.422485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:40.422485 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.422485 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:40.422485 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.422485 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.422485 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:40.422485 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.422485 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.424342 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.424852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:40.565453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:40.566616 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:40.566953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:40.567235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.568001 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:40.568001 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.568001 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:40.568001 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.568001 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.568001 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:40.568001 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.568001 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.569820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.570772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:40.587613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:40.587995 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:40.588478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:40.588753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.589434 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:40.589434 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.589434 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:40.589434 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.589434 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.589434 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:40.589434 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.589434 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:40.590562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:40.591332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:40.591332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.591332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:40.591332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.591332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.591332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:40.591332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.591332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:40.592137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.593131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.594038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:40.594038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.594038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:40.633824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:40.634382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:40.634959 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:40.635368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.636117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:40.636117 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.636117 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:40.636117 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.636117 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.636117 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:40.636117 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.636117 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:40.637470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.638211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:40.639176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.639968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:40.641224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.641224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:40.676439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:40.677101 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:40.677574 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:40.678036 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:40.678256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.679174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:40.679174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.679174 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:40.679174 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.679174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.679174 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:40.679174 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.679174 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.680766 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:40.681633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:45.382321 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:45.383480 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:45.412320 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:45.412704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.413618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:45.413618 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.413618 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:45.413618 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.413618 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.413618 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:45.413618 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.413618 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.414476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.415748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:45.566286 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:45.567213 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:45.567412 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:45.567630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.568148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:45.568148 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.568148 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:45.568148 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.568148 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.568148 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:45.568148 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.568148 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.569395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.569855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:45.589161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:45.589161 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:45.589683 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:45.590077 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.590841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:45.590841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.590841 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:45.590841 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.590841 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.590841 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:45.590841 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.590841 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:45.591744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:45.592327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:45.592327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.592327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:45.592327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.592327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.592327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:45.592327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.592327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:45.592825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.593488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.594248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:45.594248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.594248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:45.634326 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:45.634680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:45.635314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:45.635614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.636609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:45.636609 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.636609 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:45.636609 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.636609 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.636609 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:45.636609 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.636609 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:45.636904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.637435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:45.637725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.638244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:45.638829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.638829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:45.677063 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:45.678000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:45.678720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:45.679038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:45.681066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.682847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:45.682847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.682847 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:45.682847 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.682847 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.682847 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:45.682847 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.682847 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.683594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:45.684339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:50.382950 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:50.383869 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:50.414116 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:50.414371 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.414865 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:50.414865 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.414865 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:50.414865 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.414865 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.414865 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:50.414865 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.414865 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.416079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.416551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:50.566990 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:50.568093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:50.568301 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:50.568429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.569084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:50.569084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.569084 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:50.569084 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.569084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.569084 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:50.569084 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.569084 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.570744 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.571140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:50.588504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:50.591071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:50.591700 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:50.591877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.592426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:50.592426 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.592426 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:50.592426 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.592426 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.592426 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:50.592426 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.592426 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:50.593485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:50.593826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:50.593826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.593826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:50.593826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.593826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.593826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:50.593826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.593826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:50.594735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.595243 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.595596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:50.595596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.595596 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:50.634742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:50.635147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:50.635774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:50.636073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.636679 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:50.636679 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.636679 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:50.636679 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.636679 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.636679 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:50.636679 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.636679 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:50.637385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.637677 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:50.638161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.638499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:50.638845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.638845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:50.677704 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:50.678496 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:50.678942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:50.679174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:50.679296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.680165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:50.680165 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.680165 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:50.680165 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.680165 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.680165 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:50.680165 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.680165 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.680942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:50.681344 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:55.383324 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:55.384247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:55.412576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:55.412840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.413821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:34:55.413821 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.413821 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:55.413821 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.413821 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.413821 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:55.413821 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.413821 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.416052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.417146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:55.568147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:55.569316 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:55.569537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:55.569693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.570370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:55.570370 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.570370 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:55.570370 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.570370 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.570370 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:55.570370 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.570370 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.572306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.572861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:55.591088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:55.591533 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:34:55.592003 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:55.592186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.592627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:34:55.592627 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.592627 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:55.592627 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.592627 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.592627 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:55.592627 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.592627 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:55.593261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.594102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:55.594904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.595537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:55.635278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:55.635668 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:55.636421 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:55.636847 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.637608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:34:55.637608 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.637608 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:55.637608 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.637608 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.637608 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:55.637608 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.637608 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:34:55.638257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:55.638955 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:55.639725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.642773 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:34:55.678347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:34:55.679380 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:34:55.679996 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:55.680507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:34:55.681215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.682392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:34:55.682392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.682392 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:34:55.682392 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.682392 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.682392 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:34:55.682392 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.682392 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.683774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:34:55.684818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:00.384139 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:00.385917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:00.418786 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:00.419388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.420489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:00.420489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.420489 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:00.420489 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.420489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.420489 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:00.420489 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.420489 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.421587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.422300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:00.568818 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:00.570724 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:00.571120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:00.571611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.572610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:00.572610 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.572610 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:00.572610 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.572610 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.572610 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:00.572610 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.572610 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.574320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.575416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:00.591645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:00.592005 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:00.592648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:00.593003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.593818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:00.593818 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.593818 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:00.593818 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.593818 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.593818 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:00.593818 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.593818 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:00.595185 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:00.596030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:00.596030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.596030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:00.596030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.596030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.596030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:00.596030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.596030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:00.596786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.597643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.598509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:00.598509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.598509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:00.635703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:00.636199 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:00.637156 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:00.637843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.638954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:00.638954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.638954 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:00.638954 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.638954 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.638954 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:00.638954 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.638954 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:00.640135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:00.641395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:00.642370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.643530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:00.679116 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:00.679621 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:00.680180 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:00.680466 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:00.680585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.681206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:00.681206 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.681206 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:00.681206 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.681206 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.681206 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:00.681206 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.681206 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.682138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:00.682826 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:05.384842 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:05.386152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:05.421924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:05.422446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.423311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:05.423311 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.423311 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:05.423311 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.423311 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.423311 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:05.423311 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.423311 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.424492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.425328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:05.570264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:05.571607 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:05.571948 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:05.572233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.573119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:05.573119 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.573119 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:05.573119 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.573119 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.573119 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:05.573119 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.573119 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.574333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.575231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:05.592243 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:05.592635 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:05.593194 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:05.593498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.594265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:05.594265 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.594265 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:05.594265 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.594265 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.594265 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:05.594265 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.594265 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:05.595469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:05.596361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:05.596361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.596361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:05.596361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.596361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.596361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:05.596361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.596361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:05.596923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.597673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.598489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:05.598489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.598489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:05.636175 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:05.636543 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:05.637081 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:05.637429 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.638284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:05.638284 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.638284 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:05.638284 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.638284 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.638284 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:05.638284 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.638284 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:05.639571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.640015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:05.641302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.641756 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:05.642341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.642341 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:05.679588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:05.680120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:05.680660 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:05.681051 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:05.681283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.682092 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:05.682092 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.682092 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:05.682092 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.682092 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.682092 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:05.682092 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.682092 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.683630 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:05.684516 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:10.385443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:10.387205 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:10.422353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:10.422735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.423648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:10.423648 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.423648 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:10.423648 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.423648 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.423648 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:10.423648 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.423648 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.424796 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.426179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:10.571028 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:10.572883 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:10.573270 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:10.573524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.574413 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:10.574413 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.574413 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:10.574413 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.574413 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.574413 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:10.574413 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.574413 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.576448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.577446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:10.592656 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:10.593002 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:10.593431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:10.593735 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.594389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:10.594389 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.594389 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:10.594389 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.594389 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.594389 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:10.594389 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.594389 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:10.595159 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:10.596035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:10.596035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.596035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:10.596035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.596035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.596035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:10.596035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.596035 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:10.596807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.597797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.598964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:10.598964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.598964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:10.636736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:10.637434 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:10.638334 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:10.638740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.639635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:10.639635 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.639635 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:10.639635 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.639635 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.639635 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:10.639635 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.639635 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:10.640934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.641624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:10.642531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.643780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:10.644808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.644808 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:10.680294 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:10.680760 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:10.681456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:10.681871 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:10.682146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.683033 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:10.683033 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.683033 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:10.683033 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.683033 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.683033 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:10.683033 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.683033 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.684749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:10.685472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:15.386073 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:15.387085 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:15.413566 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:15.413997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.415235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:15.415235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.415235 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:15.415235 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.415235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.415235 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:15.415235 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.415235 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.416281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.417186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:15.571782 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:15.573116 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:15.573457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:15.573641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.574541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:15.574541 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.574541 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:15.574541 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.574541 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.574541 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:15.574541 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.574541 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.576230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.577086 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:15.593164 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:15.593699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:15.594479 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:15.594813 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.595719 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:15.595719 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.595719 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:15.595719 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.595719 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.595719 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:15.595719 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.595719 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:15.597106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:15.597912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:15.597912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.597912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:15.597912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.597912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.597912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:15.597912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.597912 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:15.598684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.599442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.600253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:15.600253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.600253 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:15.637437 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:15.638001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:15.638639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:15.639052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.639886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:15.639886 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.639886 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:15.639886 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.639886 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.639886 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:15.639886 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.639886 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:15.641385 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.642081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:15.643157 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.643893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:15.645135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.645135 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:15.680782 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:15.681281 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:15.681787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:15.682229 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:15.682486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.683285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:15.683285 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.683285 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:15.683285 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.683285 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.683285 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:15.683285 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.683285 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.684909 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:15.685844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:20.386661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:20.388296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:20.420699 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:20.421166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.422053 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:20.422053 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.422053 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:20.422053 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.422053 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.422053 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:20.422053 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.422053 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.423248 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.424259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:20.572521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:20.573740 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:20.574084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:20.574318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.575168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:20.575168 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.575168 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:20.575168 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.575168 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.575168 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:20.575168 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.575168 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.576714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.577881 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:20.593543 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:20.593883 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:20.594418 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:20.594708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.595448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:20.595448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.595448 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:20.595448 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.595448 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.595448 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:20.595448 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.595448 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:20.596623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:20.597607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:20.597607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.597607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:20.597607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.597607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.597607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:20.597607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.597607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:20.598108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.598893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.600042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:20.600042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.600042 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:20.637927 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:20.638388 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:20.638906 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:20.639694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.640333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:20.640333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.640333 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:20.640333 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.640333 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.640333 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:20.640333 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.640333 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:20.641233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.641939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:20.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.643223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:20.644181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.644181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:20.681361 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:20.681860 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:20.682486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:20.682937 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:20.683283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.684241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:20.684241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.684241 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:20.684241 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.684241 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.684241 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:20.684241 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.684241 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.685771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:20.686573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:25.387352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:25.389409 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:25.422567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:25.423076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.423563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:25.423563 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.423563 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:25.423563 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.423563 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.423563 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:25.423563 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.423563 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.424705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.425573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:25.573218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:25.574625 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:25.575088 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:25.575387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.576400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:25.576400 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.576400 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:25.576400 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.576400 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.576400 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:25.576400 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.576400 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.577713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.578655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:25.593913 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:25.594778 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:25.595465 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:25.595864 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.596734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:25.596734 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.596734 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:25.596734 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.596734 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.596734 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:25.596734 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.596734 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:25.598154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:25.599173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:25.599173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.599173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:25.599173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.599173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.599173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:25.599173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.599173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:25.599703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.600490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.600910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:25.600910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.600910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:25.638588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:25.639798 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:25.640533 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:25.640800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.641461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:25.641461 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.641461 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:25.641461 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.641461 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.641461 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:25.641461 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.641461 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:25.642453 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.643184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:25.643647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.644403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:25.645394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.645394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:25.681927 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:25.682638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:25.683093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:25.683363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:25.683527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.684289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:25.684289 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.684289 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:25.684289 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.684289 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.684289 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:25.684289 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.684289 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.686146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:25.687030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:30.388179 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:30.389747 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:30.391632 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:35:30.422350 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:30.422551 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.423093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:30.423093 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.423093 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:30.423093 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.423093 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.423093 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:30.423093 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.423093 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.423860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.424696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:30.573954 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:30.575340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:30.575646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:30.575932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.576958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:30.576958 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.576958 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:30.576958 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.576958 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.576958 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:30.576958 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.576958 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.578734 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.581070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:30.594609 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:30.595104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:30.595534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:30.595723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.596360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:30.596360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.596360 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:30.596360 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.596360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.596360 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:30.596360 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.596360 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:30.597548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:30.598869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:30.598869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.598869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:30.598869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.598869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.598869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:30.598869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.598869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:30.599499 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.600296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:30.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:30.639876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:30.640451 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:30.641063 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:30.641373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.642230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:30.642230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.642230 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:30.642230 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.642230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.642230 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:30.642230 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.642230 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:30.643577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.644302 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:30.645241 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.646005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:30.646872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.646872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:30.682597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:30.683201 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:30.683680 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:30.683941 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:30.684122 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.684782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:30.684782 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.684782 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:30.684782 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.684782 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.684782 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:30.684782 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.684782 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.686627 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:30.686967 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:35.388622 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:35.389835 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:35.419484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:35.419818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.420570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:35.420570 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.420570 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:35.420570 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.420570 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.420570 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:35.420570 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.420570 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.421722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.422607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:35.574588 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:35.575391 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:35.575567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:35.575714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.576546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:35.576546 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.576546 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:35.576546 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.576546 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.576546 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:35.576546 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.576546 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.577705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.578494 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:35.595115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:35.595629 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:35.596095 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:35.596376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.597078 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:35.597078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.597078 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:35.597078 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.597078 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.597078 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:35.597078 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.597078 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:35.598761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:35.599192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:35.599192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.599192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:35.599192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.599192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.599192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:35.599192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.599192 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:35.600688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.601106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.601582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:35.601582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.601582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:35.640449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:35.640686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:35.641264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:35.641602 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:35.642384 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.642384 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:35.642384 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.642384 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.642384 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:35.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.642384 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:35.643532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.644258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:35.645146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.645846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:35.646667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.646667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:35.682924 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:35.683382 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:35.684163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:35.684545 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:35.684807 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.685623 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:35.685623 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.685623 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:35.685623 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.685623 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.685623 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:35.685623 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.685623 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.686505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:35.687448 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:40.389285 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:40.390885 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:40.426250 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:40.426577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.427376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:40.427376 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.427376 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:40.427376 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.427376 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.427376 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:40.427376 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.427376 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.428076 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.428836 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:40.575406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:40.576318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:40.576592 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:40.576592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.577174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:40.577174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.577174 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:40.577174 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.577174 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.577174 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:40.577174 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.577174 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.578422 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.579277 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:40.595666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:40.596087 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:40.596478 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:40.596635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.597095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:40.597095 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.597095 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:40.597095 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.597095 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.597095 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:40.597095 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.597095 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:40.597998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:40.598513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:40.598513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.598513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:40.598513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.598513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.598513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:40.598513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.598513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:40.598944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.599481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.600021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:40.600021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.600021 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:40.641026 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:40.641489 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:40.642033 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:40.642361 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.643082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:40.643082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.643082 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:40.643082 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.643082 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.643082 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:40.643082 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.643082 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:40.643879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.644306 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:40.644934 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.645400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:40.645890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.645890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:40.683524 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:40.683923 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:40.684958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:40.684958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:40.685212 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.685672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:40.685672 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.685672 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:40.685672 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.685672 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.685672 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:40.685672 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.685672 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.686995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:40.687599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:45.389932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:45.391749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:45.426146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:45.426569 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.427493 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:45.427493 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.427493 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:45.427493 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.427493 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.427493 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:45.427493 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.427493 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.428711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.429366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:45.576013 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:45.577676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:45.577956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:45.578322 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.579345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:45.579345 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.579345 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:45.579345 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.579345 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.579345 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:45.579345 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.579345 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.580992 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.581662 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:45.595999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:45.596266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:45.596964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:45.597353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.598230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:45.598230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.598230 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:45.598230 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.598230 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.598230 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:45.598230 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.598230 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:45.599343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:45.600176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:45.600176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.600176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:45.600176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.600176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.600176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:45.600176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.600176 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:45.600765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.601531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.602377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:45.602377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.602377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:45.641540 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:45.641766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:45.642391 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:45.642814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.643610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:45.643610 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.643610 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:45.643610 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.643610 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.643610 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:45.643610 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.643610 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:45.644315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.645015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:45.646229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.647045 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:45.648164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.648164 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:45.684019 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:45.684574 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:45.685098 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:45.685386 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:45.685543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.686203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:45.686203 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.686203 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:45.686203 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.686203 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.686203 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:45.686203 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.686203 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.687674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:45.688526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:50.390711 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:50.392402 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:50.427048 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:50.427286 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.427901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:50.427901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.427901 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:50.427901 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.427901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.427901 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:50.427901 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.427901 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.428824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.429788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:50.576726 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:50.578071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:50.578416 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:50.578654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.579489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:50.579489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.579489 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:50.579489 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.579489 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.579489 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:50.579489 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.579489 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.581565 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.582388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:50.596541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:50.596958 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:50.597660 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:50.598073 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.598782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:50.598782 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.598782 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:50.598782 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.598782 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.598782 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:50.598782 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.598782 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:50.599968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:50.600771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:50.600771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.600771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:50.600771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.600771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.600771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:50.600771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.600771 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:50.601594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.602376 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.603259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:50.603259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.603259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:50.642046 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:50.642435 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:50.643104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:50.643500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.644492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:50.644492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.644492 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:50.644492 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.644492 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.644492 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:50.644492 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.644492 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:50.646017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.647050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:50.648063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.649252 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:50.650520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.650520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:50.684666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:50.685296 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:50.685888 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:50.686283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:50.686506 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.687368 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:50.687368 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.687368 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:50.687368 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.687368 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.687368 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:50.687368 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.687368 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.689210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:50.690327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:55.391438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:55.393003 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:55.427240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:55.427463 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.427957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:35:55.427957 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.427957 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:55.427957 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.427957 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.427957 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:55.427957 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.427957 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.428860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.429835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:55.577456 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:55.578836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:55.579023 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:55.579137 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.579855 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:55.579855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.579855 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:55.579855 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.579855 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.579855 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:55.579855 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.579855 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.581995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.582483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:55.597150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:55.597515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:35:55.598398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:55.598616 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.599470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:35:55.599470 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.599470 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:55.599470 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.599470 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.599470 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:55.599470 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.599470 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:55.600024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.600966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:55.601871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.602903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:55.642665 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:55.643160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:55.643727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:55.644201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.645038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:35:55.645038 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.645038 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:55.645038 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.645038 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.645038 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:55.645038 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.645038 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:35:55.646332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.647018 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:35:55.647972 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.648798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:55.649635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.649635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:35:55.685326 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:35:55.686170 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:35:55.686702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:55.687101 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:35:55.687315 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.688104 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:35:55.688104 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.688104 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:35:55.688104 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.688104 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.688104 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:35:55.688104 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.688104 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.690181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:35:55.690858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:00.392237 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:00.393453 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:00.428659 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:00.428913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.429629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:00.429629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.429629 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:00.429629 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.429629 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.429629 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:00.429629 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.429629 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.430510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.431098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:00.578223 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:00.579603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:00.579955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:00.580216 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.581037 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:00.581037 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.581037 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:00.581037 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.581037 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.581037 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:00.581037 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.581037 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.582877 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.583478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:00.597689 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:00.598195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:00.598768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:00.599064 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.599890 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:00.599890 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.599890 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:00.599890 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.599890 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.599890 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:00.599890 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.599890 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:00.601718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:00.602369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:00.602369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.602369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:00.602369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.602369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.602369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:00.602369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.602369 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:00.603850 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.604462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.605127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:00.605127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.605127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:00.643176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:00.643676 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:00.644554 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:00.645158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.646208 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:00.646208 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.646208 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:00.646208 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.646208 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.646208 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:00.646208 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.646208 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:00.647563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.648062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:00.649769 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.650699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:00.651590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.651590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:00.685935 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:00.686582 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:00.687341 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:00.687678 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:00.688050 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.688959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:00.688959 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.688959 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:00.688959 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.688959 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.688959 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:00.688959 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.688959 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.690675 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:00.691666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:05.392872 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:05.396120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:05.435200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:05.435600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.436483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:05.436483 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.436483 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:05.436483 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.436483 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.436483 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:05.436483 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.436483 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.437375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.438307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:05.579001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:05.580501 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:05.580822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:05.581363 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.581835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:05.581835 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.581835 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:05.581835 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.581835 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.581835 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:05.581835 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.581835 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.583070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.583706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:05.598336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:05.598806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:05.599700 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:05.600199 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.601098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:05.601098 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.601098 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:05.601098 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.601098 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.601098 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:05.601098 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.601098 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:05.602404 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:05.603321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:05.603321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.603321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:05.603321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.603321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.603321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:05.603321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.603321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:05.604005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.604903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.605543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:05.605543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.605543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:05.643746 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:05.644290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:05.644943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:05.645398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.646283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:05.646283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.646283 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:05.646283 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.646283 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.646283 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:05.646283 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.646283 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:05.647559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.648292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:05.649359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.650133 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:05.650925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.650925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:05.686519 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:05.687431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:05.688039 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:05.688425 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:05.688776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.689621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:05.689621 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.689621 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:05.689621 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.689621 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.689621 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:05.689621 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.689621 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.691846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:05.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:10.394877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:10.396143 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:10.432385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:10.432799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.433741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:10.433741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.433741 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:10.433741 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.433741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.433741 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:10.433741 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.433741 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.434951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.436202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:10.579764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:10.581312 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:10.581651 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:10.581960 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.582742 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:10.582742 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.582742 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:10.582742 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.582742 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.582742 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:10.582742 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.582742 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.584562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.585522 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:10.598806 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:10.599261 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:10.599772 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:10.600107 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.601079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:10.601079 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.601079 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:10.601079 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.601079 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.601079 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:10.601079 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.601079 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:10.602173 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:10.603008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:10.603008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.603008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:10.603008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.603008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.603008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:10.603008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.603008 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:10.603694 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.604288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.604853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:10.604853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.604853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:10.644159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:10.644691 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:10.645542 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:10.645853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.646676 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:10.646676 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.646676 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:10.646676 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.646676 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.646676 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:10.646676 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.646676 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:10.647941 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.648658 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:10.649479 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.650293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:10.651340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.651340 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:10.687004 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:10.687558 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:10.687999 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:10.688373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:10.688603 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.689381 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:10.689381 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.689381 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:10.689381 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.689381 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.689381 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:10.689381 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.689381 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.690904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:10.691745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:15.395673 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:15.397052 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:15.431192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:15.431571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.432456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:15.432456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.432456 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:15.432456 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.432456 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.432456 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:15.432456 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.432456 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.433343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.434294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:15.580541 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:15.581790 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:15.582295 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:15.582590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.583473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:15.583473 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.583473 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:15.583473 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.583473 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.583473 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:15.583473 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.583473 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.585170 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.586043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:15.599406 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:15.599742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:15.600279 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:15.600614 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.601391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:15.601391 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.601391 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:15.601391 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.601391 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.601391 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:15.601391 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.601391 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:15.602713 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:15.603544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:15.603544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.603544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:15.603544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.603544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.603544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:15.603544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.603544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:15.604599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.605353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.606288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:15.606288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.606288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:15.644721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:15.645272 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:15.645917 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:15.646349 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.647179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:15.647179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.647179 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:15.647179 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.647179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.647179 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:15.647179 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.647179 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:15.648358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.649032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:15.649965 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.650853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:15.651722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.651722 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:15.687580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:15.688165 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:15.688675 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:15.689092 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:15.689328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.690131 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:15.690131 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.690131 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:15.690131 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.690131 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.690131 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:15.690131 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.690131 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.691902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:15.692768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:20.396404 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:20.397671 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:20.429666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:20.429940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.430674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:20.430674 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.430674 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:20.430674 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.430674 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.430674 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:20.430674 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.430674 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.431505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.432257 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:20.581735 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:20.583216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:20.583628 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:20.583951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.584901 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:20.584901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.584901 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:20.584901 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.584901 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.584901 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:20.584901 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.584901 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.587211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.588251 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:20.600104 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:20.600479 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:20.600896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:20.601168 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.601778 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:20.601778 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.601778 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:20.601778 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.601778 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.601778 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:20.601778 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.601778 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:20.603127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:20.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:20.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:20.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:20.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:20.604496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.605395 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.606362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:20.606362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.606362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:20.645408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:20.645821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:20.646344 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:20.646783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.647570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:20.647570 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.647570 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:20.647570 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.647570 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.647570 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:20.647570 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.647570 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:20.648703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.649412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:20.650310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.651139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:20.651928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.651928 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:20.688102 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:20.688768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:20.689289 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:20.689641 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:20.689852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.690815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:20.690815 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.690815 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:20.690815 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.690815 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.690815 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:20.690815 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.690815 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.693274 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:20.694481 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:25.397126 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:25.398849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:25.432356 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:25.432768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.433650 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:25.433650 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.433650 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:25.433650 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.433650 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.433650 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:25.433650 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.433650 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.434622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.435469 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:25.582595 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:25.583901 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:25.584093 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:25.584242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.584884 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:25.584884 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.584884 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:25.584884 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.584884 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.584884 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:25.584884 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.584884 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.586917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.587433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:25.600504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:25.600682 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:25.600963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:25.601392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.601597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:25.601597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.601597 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:25.601597 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.601597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.601597 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:25.601597 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.601597 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:25.602328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:25.603055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:25.603055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.603055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:25.603055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.603055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.603055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:25.603055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.603055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:25.603414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.604049 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.604829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:25.604829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.604829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:25.645920 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:25.646551 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:25.647208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:25.647498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.648325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:25.648325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.648325 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:25.648325 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.648325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.648325 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:25.648325 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.648325 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:25.649612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.650272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:25.651325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.652017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:25.652814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.652814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:25.688631 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:25.689347 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:25.689804 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:25.690184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:25.690400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.691203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:25.691203 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.691203 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:25.691203 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.691203 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.691203 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:25.691203 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.691203 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.692999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:25.693704 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:30.397891 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:30.399418 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:30.401021 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:36:30.432505 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:30.433279 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.433888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:30.433888 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.433888 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:30.433888 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.433888 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.433888 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:30.433888 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.433888 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.435267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.436161 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:30.583843 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:30.585235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:30.585585 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:30.585832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.586995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:30.586995 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.586995 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:30.586995 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.586995 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.586995 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:30.586995 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.586995 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.588903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.589492 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:30.600859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:30.601166 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:30.601583 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:30.601852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.602605 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:30.602605 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.602605 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:30.602605 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.602605 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.602605 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:30.602605 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.602605 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:30.603821 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:30.604631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:30.604631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.604631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:30.604631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.604631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.604631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:30.604631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.604631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:30.605817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.606384 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.607231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:30.607231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.607231 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:30.646590 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:30.647216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:30.647775 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:30.648258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.649226 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:30.649226 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.649226 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:30.649226 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.649226 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.649226 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:30.649226 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.649226 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:30.650649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.651318 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:30.652472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.653269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:30.654106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.654106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:30.690012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:30.690831 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:30.691399 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:30.691879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:30.692293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.693093 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:30.693093 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.693093 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:30.693093 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.693093 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.693093 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:30.693093 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.693093 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:30.695875 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:35.398607 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:35.400262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:35.432173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:35.432561 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.433454 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:35.433454 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.433454 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:35.433454 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.433454 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.433454 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:35.433454 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.433454 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.434671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.436108 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:35.584385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:35.585255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:35.585482 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:35.585633 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.586290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:35.586290 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.586290 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:35.586290 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.586290 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.586290 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:35.586290 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.586290 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.587748 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.589062 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:35.601325 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:35.601639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:35.602065 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:35.602324 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.602803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:35.602803 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.602803 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:35.602803 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.602803 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.602803 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:35.602803 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.602803 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:35.604080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:35.604797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:35.604797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.604797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:35.604797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.604797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.604797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:35.604797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.604797 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:35.605740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.607228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.608288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:35.608288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.608288 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:35.646961 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:35.647681 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:35.648239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:35.648723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.649532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:35.649532 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.649532 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:35.649532 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.649532 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.649532 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:35.649532 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.649532 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:35.651210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.652046 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:35.652866 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.653665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:35.654559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.654559 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:35.690603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:35.691302 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:35.691943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:35.692231 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:35.692475 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.693354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:35.693354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.693354 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:35.693354 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.693354 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.693354 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:35.693354 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.693354 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.695171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:35.695922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:40.399224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:40.400871 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:40.440189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:40.440925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.441329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:40.441329 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.441329 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:40.441329 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.441329 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.441329 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:40.441329 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.441329 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.442823 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.443297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:40.584864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:40.586302 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:40.586604 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:40.586729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.587558 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:40.587558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.587558 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:40.587558 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.587558 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.587558 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:40.587558 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.587558 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.589162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.590326 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:40.601771 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:40.602648 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:40.603256 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:40.603528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.604325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:40.604325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.604325 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:40.604325 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.604325 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.604325 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:40.604325 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.604325 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:40.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:40.606759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:40.606759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.606759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:40.606759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.606759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.606759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:40.606759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.606759 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:40.607745 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.608478 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.609098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:40.609098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.609098 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:40.647575 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:40.648342 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:40.648991 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:40.649446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.650232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:40.650232 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.650232 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:40.650232 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.650232 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.650232 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:40.650232 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.650232 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:40.651582 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.652244 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:40.653284 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.654117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:40.654919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.654919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:40.691200 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:40.691743 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:40.692353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:40.692748 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:40.692959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.693898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:40.693898 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.693898 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:40.693898 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.693898 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.693898 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:40.693898 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.693898 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.695526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:40.696441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:45.400144 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:45.401717 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:45.436759 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:45.437203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.438103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:45.438103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.438103 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:45.438103 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.438103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.438103 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:45.438103 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.438103 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.439266 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.439878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:45.585609 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:45.586567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:45.586896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:45.586896 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.587465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:45.587465 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.587465 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:45.587465 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.587465 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.587465 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:45.587465 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.587465 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.588500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.589087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:45.602408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:45.602821 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:45.603184 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:45.603452 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.604084 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:45.604084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.604084 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:45.604084 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.604084 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.604084 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:45.604084 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.604084 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:45.604867 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:45.605358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:45.605358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.605358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:45.605358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.605358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.605358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:45.605358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.605358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:45.605767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.606215 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.606710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:45.606710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.606710 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:45.648013 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:45.648652 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:45.649198 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:45.649529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.650356 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:45.650356 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.650356 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:45.650356 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.650356 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.650356 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:45.650356 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.650356 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:45.651691 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.652379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:45.653498 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.654298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:45.655121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.655121 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:45.691764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:45.692518 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:45.692998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:45.693549 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:45.693549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.694473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:45.694473 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.694473 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:45.694473 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.694473 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.694473 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:45.694473 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.694473 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.696751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:45.697626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:50.400879 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:50.402442 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:50.434750 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:50.436290 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.437276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:50.437276 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.437276 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:50.437276 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.437276 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.437276 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:50.437276 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.437276 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.438497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.439610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:50.586379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:50.587479 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:50.587769 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:50.587943 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.588645 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:50.588645 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.588645 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:50.588645 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.588645 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.588645 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:50.588645 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.588645 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.590228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.591145 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:50.602856 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:50.603111 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:50.603389 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:50.603576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:50.604058 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.604058 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:50.604058 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.604058 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.604058 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:50.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.604058 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:50.604794 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:50.605681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:50.605681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.605681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:50.605681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.605681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.605681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:50.605681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.605681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:50.606134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.606893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.607819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:50.607819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.607819 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:50.648485 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:50.649151 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:50.649688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:50.650094 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.650869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:50.650869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.650869 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:50.650869 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.650869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.650869 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:50.650869 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.650869 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:50.652291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.653063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:50.653873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.654649 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:50.655465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.655465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:50.692290 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:50.692938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:50.693487 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:50.693864 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:50.694114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.695002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:50.695002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.695002 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:50.695002 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.695002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.695002 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:50.695002 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.695002 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.696784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:50.697714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:55.401546 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:55.403053 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:55.446360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:55.446663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.447657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:36:55.447657 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.447657 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:55.447657 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.447657 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.447657 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:55.447657 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.447657 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.449521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.450706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:55.587132 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:55.588300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:55.588654 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:55.588910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.589894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:55.589894 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.589894 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:55.589894 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.589894 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.589894 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:55.589894 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.589894 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.591708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.592861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:55.603333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:55.603896 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:36:55.604532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:55.604840 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.605606 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:36:55.605606 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.605606 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:55.605606 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.605606 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.605606 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:55.605606 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.605606 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:55.606834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:55.607673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:55.607673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.607673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:55.607673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.607673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.607673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:55.607673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.607673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:36:55.608364 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.609071 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.609873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:55.609873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.609873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:55.648972 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:55.649381 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:55.649877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:55.650305 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.650997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:36:55.650997 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.650997 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:55.650997 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.650997 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.650997 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:55.650997 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.650997 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:36:55.651893 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.652737 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:36:55.653138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.653805 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:55.654595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.654595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:36:55.692735 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:36:55.693336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:36:55.693829 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:55.694243 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:36:55.694800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.695249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:36:55.695249 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.695249 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:36:55.695249 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.695249 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.695249 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:36:55.695249 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.695249 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.696511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:36:55.697671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:00.402248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:00.404219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:00.439742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:00.440020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.440776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:00.440776 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.440776 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:00.440776 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.440776 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.440776 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:00.440776 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.440776 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.441825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.442724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:00.587755 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:00.589016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:00.589323 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:00.589511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.590307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:00.590307 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.590307 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:00.590307 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.590307 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.590307 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:00.590307 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.590307 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.592269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.593081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:00.603764 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:00.603965 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:00.604467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:00.604681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.605144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:00.605144 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.605144 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:00.605144 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.605144 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.605144 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:00.605144 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.605144 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:00.605827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.606728 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:00.607519 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.608527 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:00.649413 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:00.650055 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:00.650666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:00.651066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.651872 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:00.651872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.651872 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:00.651872 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.651872 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.651872 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:00.651872 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.651872 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:00.653220 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.653874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:00.655004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.655780 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:00.656566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.656566 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:00.693314 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:00.693861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:00.694401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:00.694817 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:00.695100 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.695964 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:00.695964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.695964 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:00.695964 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.695964 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.695964 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:00.695964 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.695964 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.699229 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:00.699954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:05.403102 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:05.404666 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:05.437266 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:05.437416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.437861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:05.437861 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.437861 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:05.437861 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.437861 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.437861 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:05.437861 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.437861 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.438638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.439631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:05.588443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:05.589723 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:05.589953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:05.590130 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.590779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:05.590779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.590779 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:05.590779 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.590779 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.590779 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:05.590779 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.590779 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.592672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.593198 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:05.604233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:05.604525 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:05.604875 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:05.605103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.605552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:05.605552 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.605552 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:05.605552 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.605552 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.605552 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:05.605552 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.605552 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:05.606294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:05.607103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:05.607103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.607103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:05.607103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.607103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.607103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:05.607103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.607103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:05.607651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.608400 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.609299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:05.609299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.609299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:05.649932 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:05.650513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:05.651449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:05.651994 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.652767 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:05.652767 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.652767 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:05.652767 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.652767 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.652767 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:05.652767 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.652767 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:05.653708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.654428 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:05.654816 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.655641 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:05.656531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.656531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:05.693833 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:05.694373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:05.694824 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:05.695224 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:05.695451 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.696265 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:05.696265 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.696265 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:05.696265 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.696265 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.696265 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:05.696265 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.696265 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.697779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:05.698666 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:10.403732 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:10.405288 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:10.438278 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:10.438567 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.438825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:10.438825 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.438825 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:10.438825 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.438825 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.438825 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:10.438825 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.438825 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.439587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.440635 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:10.589154 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:10.590147 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:10.590398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:10.590592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.591237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:10.591237 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.591237 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:10.591237 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.591237 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.591237 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:10.591237 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.591237 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.592533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.593388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:10.604630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:10.605001 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:10.605556 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:10.605858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.606768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:10.606768 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.606768 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:10.606768 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.606768 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.606768 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:10.606768 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.606768 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:10.608298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:10.609238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:10.609238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.609238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:10.609238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.609238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.609238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:10.609238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.609238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:10.609827 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.611119 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.612040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:10.612040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.612040 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:10.650576 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:10.651280 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:10.652014 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:10.652509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.653403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:10.653403 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.653403 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:10.653403 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.653403 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.653403 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:10.653403 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.653403 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:10.654835 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.655532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:10.656548 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.657297 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:10.658106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.658106 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:10.694432 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:10.695006 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:10.695880 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:10.696417 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:10.696726 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.697688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:10.697688 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.697688 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:10.697688 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.697688 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.697688 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:10.697688 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.697688 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.699500 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:10.700444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:15.404351 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:15.405708 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:15.447432 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:15.447927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.448853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:15.448853 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.448853 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:15.448853 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.448853 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.448853 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:15.448853 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.448853 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.450136 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.451293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:15.589832 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:15.591241 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:15.591584 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:15.591768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.592647 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:15.592647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.592647 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:15.592647 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.592647 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.592647 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:15.592647 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.592647 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.593814 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.594684 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:15.605484 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:15.606385 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:15.607050 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:15.607346 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.608116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:15.608116 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.608116 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:15.608116 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.608116 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.608116 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:15.608116 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.608116 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:15.609693 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:15.610555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:15.610555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.610555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:15.610555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.610555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.610555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:15.610555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.610555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:15.611201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.611895 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.612789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:15.612789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.612789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:15.651208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:15.651727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:15.652212 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:15.652439 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.653116 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:15.653116 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.653116 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:15.653116 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.653116 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.653116 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:15.653116 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.653116 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:15.654417 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.655148 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:15.655659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.656436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:15.657300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.657300 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:15.694925 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:15.695445 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:15.695953 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:15.696312 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:15.696540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.697327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:15.697327 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.697327 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:15.697327 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.697327 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.697327 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:15.697327 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.697327 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.699153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:15.700165 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:20.405012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:20.406505 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:20.441110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:20.441416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.442090 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:20.442090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.442090 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:20.442090 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.442090 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.442090 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:20.442090 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.442090 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.443330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.443654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:20.590283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:20.591736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:20.592011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:20.592258 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.592802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:20.592802 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.592802 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:20.592802 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.592802 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.592802 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:20.592802 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.592802 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.593942 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.594812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:20.606020 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:20.606724 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:20.607377 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:20.607671 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.608437 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:20.608437 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.608437 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:20.608437 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.608437 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.608437 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:20.608437 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.608437 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:20.609762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:20.610838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:20.610838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.610838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:20.610838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.610838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.610838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:20.610838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.610838 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:20.611380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.612144 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.613059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:20.613059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.613059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:20.651695 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:20.652341 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:20.653152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:20.653537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.654360 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:20.654360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.654360 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:20.654360 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.654360 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.654360 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:20.654360 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.654360 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:20.655966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.656786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:20.657706 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.659048 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:20.660497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.660497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:20.695527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:20.696124 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:20.696733 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:20.697146 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:20.697365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.698128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:20.698128 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.698128 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:20.698128 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.698128 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.698128 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:20.698128 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.698128 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.699330 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:20.699856 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:25.405702 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:25.406930 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:25.436469 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:25.436880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.437746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:25.437746 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.437746 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:25.437746 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.437746 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.437746 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:25.437746 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.437746 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.438607 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.439415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:25.591046 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:25.592419 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:25.592645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:25.592788 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.593507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:25.593507 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.593507 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:25.593507 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.593507 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.593507 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:25.593507 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.593507 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.595621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.596615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:25.606774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:25.607686 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:25.608433 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:25.609031 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.609553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:25.609553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.609553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:25.609553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.609553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.609553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:25.609553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.609553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:25.611227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:25.612357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:25.612357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.612357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:25.612357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.612357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.612357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:25.612357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.612357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:25.613006 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.614129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.615012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:25.615012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.615012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:25.652394 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:25.652822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:25.653360 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:25.653702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.654445 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:25.654445 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.654445 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:25.654445 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.654445 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.654445 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:25.654445 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.654445 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:25.655663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.656405 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:25.657382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.658233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:25.659262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.659262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:25.696103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:25.696532 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:25.696918 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:25.697307 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:25.697517 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.698217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:25.698217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.698217 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:25.698217 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.698217 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.698217 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:25.698217 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.698217 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.699746 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:25.700532 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:30.406597 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:30.408232 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:30.409916 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:37:30.444429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:30.444806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.445843 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:30.445843 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.445843 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:30.445843 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.445843 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.445843 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:30.445843 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.445843 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.446998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.448246 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:30.592444 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:30.593785 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:30.594196 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:30.594446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.595373 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:30.595373 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.595373 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:30.595373 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.595373 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.595373 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:30.595373 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.595373 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.596609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.597114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:30.607560 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:30.608235 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:30.608671 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:30.608961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.609681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:30.609681 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.609681 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:30.609681 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.609681 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.609681 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:30.609681 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.609681 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:30.610563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.611521 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:30.612406 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.613626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:30.653017 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:30.653561 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:30.654095 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:30.654483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.655235 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:30.655235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.655235 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:30.655235 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.655235 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.655235 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:30.655235 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.655235 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:30.656508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.657308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:30.658254 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.659261 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:30.660416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.660416 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:30.696510 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:30.697238 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:30.697787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:30.698176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:30.698396 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.699210 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:30.699210 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.699210 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:30.699210 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.699210 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.699210 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:30.699210 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.699210 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.700599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:30.701592 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:35.407410 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:35.408887 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:35.447233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:35.447587 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.448415 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:35.448415 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.448415 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:35.448415 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.448415 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.448415 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:35.448415 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.448415 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.449299 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.450471 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:35.593300 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:35.594544 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:35.594736 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:35.594910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.595637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:35.595637 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.595637 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:35.595637 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.595637 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.595637 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:35.595637 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.595637 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.597442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.598530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:35.608090 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:35.608734 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:35.609233 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:35.609464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.610069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:35.610069 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.610069 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:35.610069 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.610069 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.610069 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:35.610069 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.610069 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:35.611089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.612236 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:35.613283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.614223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:35.653563 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:35.654513 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:35.655086 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:35.655390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.656087 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:35.656087 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.656087 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:35.656087 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.656087 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.656087 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:35.656087 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.656087 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:35.657538 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.658134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:35.659204 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.660074 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:35.660939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.660939 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:35.697119 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:35.697928 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:35.698685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:35.699148 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:35.699476 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.700195 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:35.700195 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.700195 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:35.700195 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.700195 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.700195 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:35.700195 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.700195 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.701225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:35.701550 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:40.408287 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:40.409853 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:40.441889 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:40.442403 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.443496 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:40.443496 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.443496 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:40.443496 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.443496 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.443496 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:40.443496 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.443496 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.444389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.444958 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:40.594218 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:40.595252 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:40.595472 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:40.595611 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.596267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:40.596267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.596267 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:40.596267 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.596267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.596267 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:40.596267 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.596267 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.597141 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.597931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:40.608603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:40.608841 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:40.609299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:40.609574 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.610334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:40.610334 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.610334 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:40.610334 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.610334 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.610334 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:40.610334 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.610334 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:40.611667 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:40.612673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:40.612673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.612673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:40.612673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.612673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.612673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:40.612673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.612673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:40.613223 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.613940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.614844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:40.614844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.614844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:40.654177 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:40.654661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:40.655204 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:40.655571 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.656228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:40.656228 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.656228 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:40.656228 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.656228 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.656228 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:40.656228 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.656228 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:40.656889 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:40.657681 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:40.658435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.659205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:40.697791 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:40.698369 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:40.698823 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:40.699274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:40.699564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.700316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:40.700316 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.700316 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:40.700316 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.700316 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.700316 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:40.700316 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.700316 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.701961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:40.702804 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:45.408876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:45.410589 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:45.444076 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:45.444541 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.445398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:45.445398 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.445398 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:45.445398 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.445398 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.445398 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:45.445398 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.445398 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.446402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.447456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:45.594943 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:45.595863 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:45.596428 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:45.596577 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.597483 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:45.597483 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.597483 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:45.597483 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.597483 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.597483 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:45.597483 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.597483 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.598280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.599473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:45.609078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:45.609468 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:45.610115 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:45.610466 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.611139 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:45.611139 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.611139 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:45.611139 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.611139 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.611139 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:45.611139 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.611139 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:45.611968 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:45.612489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:45.612489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.612489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:45.612489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.612489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.612489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:45.612489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.612489 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:45.613179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.613910 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.614703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:45.614703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.614703 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:45.654745 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:45.655283 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:45.655849 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:45.656271 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.657154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:45.657154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.657154 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:45.657154 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.657154 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.657154 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:45.657154 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.657154 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:45.658488 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.659354 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:45.660335 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.661138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:45.662292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.662292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:45.698341 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:45.698884 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:45.699618 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:45.700055 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:45.700307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.701263 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:45.701263 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.701263 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:45.701263 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.701263 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.701263 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:45.701263 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.701263 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.702923 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:45.703789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:50.409520 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:50.411136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:50.443559 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:50.444069 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.445000 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:50.445000 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.445000 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:50.445000 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.445000 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.445000 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:50.445000 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.445000 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.446129 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.446705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:50.595565 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:50.596865 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:50.597387 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:50.597705 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.598621 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:50.598621 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.598621 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:50.598621 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.598621 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.598621 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:50.598621 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.598621 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.600911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.601848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:50.609515 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:50.609839 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:50.610474 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:50.610818 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.611612 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:50.611612 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.611612 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:50.611612 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.611612 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.611612 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:50.611612 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.611612 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:50.612874 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:50.613786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:50.613786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.613786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:50.613786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.613786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.613786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:50.613786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.613786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:50.614473 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.616255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:50.616255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.616255 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:50.655198 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:50.655688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:50.656249 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:50.656690 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.657711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:50.657711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.657711 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:50.657711 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.657711 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.657711 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:50.657711 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.657711 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:50.658925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.659659 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:50.660736 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.661508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:50.661995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.661995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:50.698857 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:50.699294 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:50.699877 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:50.700281 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:50.700509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.701436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:50.701436 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.701436 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:50.701436 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.701436 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.701436 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:50.701436 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.701436 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.702966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:50.703995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:55.410264 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:55.412007 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:55.447226 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:55.447552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.448665 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:37:55.448665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.448665 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:55.448665 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.448665 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.448665 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:55.448665 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.448665 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.449505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:55.596293 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:55.597554 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:55.597784 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:55.597953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.598740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:55.598740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.598740 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:55.598740 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.598740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.598740 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:55.598740 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.598740 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.600485 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.601055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:55.609993 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:55.610538 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:37:55.611038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:55.611267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.611869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:37:55.611869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.611869 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:55.611869 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.611869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.611869 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:55.611869 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.611869 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:55.612781 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:55.613374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:55.613374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.613374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:55.613374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.613374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.613374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:55.613374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.613374 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:37:55.613779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.614283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.614798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:55.614798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.614798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:55.655800 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:55.656407 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:55.657251 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:55.657724 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.658674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:37:55.658674 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.658674 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:55.658674 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.658674 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.658674 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:55.658674 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.658674 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:37:55.660058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.660832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:37:55.661351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.662242 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:55.663059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.663059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:37:55.699367 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:37:55.700141 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:37:55.700967 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:55.701258 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:37:55.701414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.702624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:37:55.702624 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.702624 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:37:55.702624 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.702624 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.702624 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:37:55.702624 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.702624 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.704472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:37:55.705353 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:00.411042 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:00.412727 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:00.447525 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:00.447765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.448435 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:00.448435 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.448435 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:00.448435 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.448435 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.448435 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:00.448435 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.448435 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.449595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.450531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:00.597000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:00.598149 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:00.598454 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:00.598720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.599486 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:00.599486 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.599486 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:00.599486 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.599486 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.599486 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:00.599486 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.599486 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.600537 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.601120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:00.610486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:00.611024 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:00.611638 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:00.611999 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.612714 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:00.612714 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.612714 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:00.612714 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.612714 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.612714 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:00.612714 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.612714 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:00.614068 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:00.615276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:00.615276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.615276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:00.615276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.615276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.615276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:00.615276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.615276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:00.615751 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.616564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.617464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:00.617464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.617464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:00.656447 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:00.656916 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:00.657449 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:00.657815 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.658673 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:00.658673 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.658673 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:00.658673 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.658673 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.658673 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:00.658673 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.658673 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:00.660365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.661089 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:00.661990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.662779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:00.663615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.663615 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:00.700134 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:00.700611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:00.701174 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:00.701530 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:00.701754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.702657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:00.702657 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.702657 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:00.702657 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.702657 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.702657 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:00.702657 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.702657 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.704576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:00.705464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:05.411895 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:05.413485 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:05.445940 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:05.446386 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.447293 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:05.447293 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.447293 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:05.447293 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.447293 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.447293 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:05.447293 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.447293 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.448355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.449282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:05.597662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:05.598993 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:05.599357 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:05.599590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.600419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:05.600419 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.600419 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:05.600419 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.600419 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.600419 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:05.600419 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.600419 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.602153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.602931 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:05.610898 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:05.611486 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:05.612165 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:05.612563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.613553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:05.613553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.613553 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:05.613553 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.613553 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.613553 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:05.613553 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.613553 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:05.614715 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:05.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:05.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:05.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:05.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.615325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:05.616292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.616753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.617629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:05.617629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.617629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:05.656926 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:05.657521 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:05.658049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:05.658420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.659382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:05.659382 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.659382 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:05.659382 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.659382 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.659382 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:05.659382 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.659382 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:05.660333 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.661233 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:05.661733 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.662608 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:05.663511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.663511 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:05.700605 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:05.701353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:05.701836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:05.702214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:05.702433 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.703219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:05.703219 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.703219 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:05.703219 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.703219 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.703219 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:05.703219 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.703219 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.704880 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:05.705799 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:10.412516 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:10.414333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:10.447245 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:10.447654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.448595 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:10.448595 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.448595 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:10.448595 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.448595 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.448595 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:10.448595 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.448595 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.449487 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.450456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:10.598401 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:10.599718 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:10.600124 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:10.600365 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.601272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:10.601272 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.601272 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:10.601272 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.601272 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.601272 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:10.601272 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.601272 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.603063 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.603820 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:10.611439 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:10.611822 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:10.612315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:10.612590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.613313 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:10.613313 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.613313 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:10.613313 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.613313 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.613313 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:10.613313 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.613313 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:10.614754 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:10.615545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:10.615545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.615545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:10.615545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.615545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.615545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:10.615545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.615545 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:10.616301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.617015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.617783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:10.617783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.617783 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:10.657458 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:10.658192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:10.658752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:10.659095 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.659763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:10.659763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.659763 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:10.659763 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.659763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.659763 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:10.659763 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.659763 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:10.660685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.661348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:10.661774 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.662503 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:10.663327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.663327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:10.701192 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:10.701698 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:10.702094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:10.702441 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:10.702654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.703402 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:10.703402 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.703402 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:10.703402 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.703402 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.703402 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:10.703402 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.703402 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.704971 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:10.705765 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:15.413333 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:15.414443 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:15.446210 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:15.446632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.447661 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:15.447661 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.447661 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:15.447661 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.447661 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.447661 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:15.447661 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.447661 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.448852 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.449430 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:15.599071 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:15.600343 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:15.600611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:15.600776 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.601618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:15.601618 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.601618 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:15.601618 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.601618 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.601618 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:15.601618 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.601618 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.603217 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.604055 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:15.611767 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:15.612139 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:15.612621 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:15.612832 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.613375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:15.613375 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.613375 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:15.613375 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.613375 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.613375 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:15.613375 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.613375 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:15.614329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.615225 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:15.616030 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.616903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:15.658118 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:15.658845 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:15.659340 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:15.659725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.660631 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:15.660631 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.660631 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:15.660631 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.660631 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.660631 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:15.660631 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.660631 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:15.662127 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.662542 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:15.663858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.664388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:15.664845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.664845 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:15.701743 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:15.702458 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:15.703197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:15.703580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:15.703812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.704782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:15.704782 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.704782 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:15.704782 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.704782 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.704782 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:15.704782 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.704782 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.706624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:15.707370 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:20.414098 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:20.415553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:20.450299 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:20.450564 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.451232 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:20.451232 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.451232 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:20.451232 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.451232 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.451232 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:20.451232 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.451232 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.452462 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.453534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:20.599814 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:20.601054 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:20.601438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:20.601806 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.602560 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:20.602560 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.602560 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:20.602560 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.602560 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.602560 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:20.602560 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.602560 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.604366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.604997 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:20.612171 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:20.612787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:20.613463 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:20.613738 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.614446 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:20.614446 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.614446 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:20.614446 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.614446 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.614446 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:20.614446 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.614446 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:20.615533 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:20.616249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:20.616249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.616249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:20.616249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.616249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.616249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:20.616249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.616249 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:20.616543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.617088 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.617869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:20.617869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.617869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:20.658698 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:20.659265 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:20.659876 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:20.660303 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.661113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:20.661113 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.661113 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:20.661113 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.661113 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.661113 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:20.661113 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.661113 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:20.662510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.663003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:20.664224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.664594 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:20.664962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.664962 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:20.702259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:20.703012 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:20.703617 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:20.704060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:20.704358 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.705187 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:20.705187 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.705187 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:20.705187 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.705187 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.705187 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:20.705187 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.705187 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.707179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:20.707953 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:24.809182 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:24.821046 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}}
Mar 26 01:38:24.821183 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105681) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}}
Mar 26 01:38:24.821308 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Fixing resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:38:24.858173 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, resource={'id': '7c327d4f-b960-49ee-8c2b-fc5ec1b488ad', 'name': 'private', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['02d96a4b-4c93-4b95-ad0f-f11d085f0699', '0503b55a-1b69-49b5-b418-16ba37be90e9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T00:58:10Z', 'updated_at': '2026-03-26T00:58:13Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4449}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.858600 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7c327d4f-b960-49ee-8c2b-fc5ec1b488ad, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.859189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6fe31530-ff9f-476b-aea1-4804059b76cd, external_ids_update=None, columns={'tag': 4449}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.859444 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.860366 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Traceback (most recent call last):
Mar 26 01:38:24.860366 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:24.860366 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:24.860366 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:24.860366 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:24.860366 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:24.860366 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.860366 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Failed to fix resource 7c327d4f-b960-49ee-8c2b-fc5ec1b488ad (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:24.861624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:38:24.862547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:24.863285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:38:24.863285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.863285 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:38:24.863285 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Fixing resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:38:24.895060 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-db438906-c74f-4dc8-883c-6e56962e551b, resource={'id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'name': 'public', 'tenant_id': '2a82c12cc6e34337bbe64e30544913a3', 'project_id': '2a82c12cc6e34337bbe64e30544913a3', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['1c3e4583-6e63-4953-8050-3341662a5ec6', '23682989-62bf-4e07-a74f-0cb01077d5ac'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-03-26T00:58:16Z', 'updated_at': '2026-03-26T00:58:25Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.895315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-db438906-c74f-4dc8-883c-6e56962e551b, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.895737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-200618ec-0540-41b0-9429-dc7dfc74abb0, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.895952 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.897378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Traceback (most recent call last):
Mar 26 01:38:24.897378 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:24.897378 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:24.897378 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:24.897378 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:24.897378 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:24.897378 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.897378 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Failed to fix resource db438906-c74f-4dc8-883c-6e56962e551b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:24.898626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:38:24.899338 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:24.899966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:38:24.899966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.899966 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:38:24.899966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Fixing resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:38:24.925635 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, resource={'id': 'f7e819e4-7101-412b-b047-baa5e16b2a52', 'name': 'admin_net', 'tenant_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'project_id': 'ddc3626e01ae43248e1d22ec992fe4ef', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f2e900ee-4d88-4151-8e50-661745950632'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:14Z', 'updated_at': '2026-03-26T01:02:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 4803}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.926011 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f7e819e4-7101-412b-b047-baa5e16b2a52, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.926514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d43d91f1-f24a-411f-bae7-af85048f95b9, external_ids_update=None, columns={'tag': 4803}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.926672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.927390 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Traceback (most recent call last):
Mar 26 01:38:24.927390 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:24.927390 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:24.927390 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:24.927390 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:24.927390 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:24.927390 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.927390 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Failed to fix resource f7e819e4-7101-412b-b047-baa5e16b2a52 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:24.928379 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:38:24.928886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:24.929512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:38:24.929512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.929512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:38:24.929512 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Fixing resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:38:24.954669 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, resource={'id': 'f481c2f9-5052-46d5-821a-6c8712ec70ec', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['329ea1d3-f656-45b1-bf23-1d927d4578ea'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-03-26T01:02:50Z', 'updated_at': '2026-03-26T01:02:51Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19903}, resource_type=networks, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.955074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f481c2f9-5052-46d5-821a-6c8712ec70ec, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.955628 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3aa5be7a-5c67-4f1a-bdf2-7bf6912e871d, external_ids_update=None, columns={'tag': 19903}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:24.955785 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.956526 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Traceback (most recent call last):
Mar 26 01:38:24.956526 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:24.956526 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:24.956526 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:24.956526 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:24.956526 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:24.956526 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.956526 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Failed to fix resource f481c2f9-5052-46d5-821a-6c8712ec70ec (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2302, in update_network
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:24.957637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:38:24.958206 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:24.958825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:38:24.958825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:24.958825 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:38:24.958825 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Fixing resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:38:25.024633 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, resource={'id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'project_id': '174e5ccfb7414601ba8d507631a4aba8', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'enable_snat': True}, 'gw_port_id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-ryt6n', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'external_fixed_ips': [{'ip_address': '172.24.5.142', 'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac'}, {'ip_address': '2001:db8::16a', 'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 3, 'tenant_id': '174e5ccfb7414601ba8d507631a4aba8'}, resource_type=routers, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.024956 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-ryt6n', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.025309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.025523 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-db438906-c74f-4dc8-883c-6e56962e551b, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "14a0aa42-40aa-4508-9af3-cffdffef9818"')], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.025709 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.026530 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Traceback (most recent call last):
Mar 26 01:38:25.026530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.026530 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:25.026530 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.026530 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.026530 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:25.026530 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.026530 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:25.026904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Unable to update router e4efd798-3c25-4ea9-bd45-fa7a05aeaef9. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Failed to fix resource e4efd798-3c25-4ea9-bd45-fa7a05aeaef9 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1609, in update_router
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Mar 26 01:38:25.028269 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1543, in update_router
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:38:25.028779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:38:25.029392 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Fixing resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports) at create/update {{(pid=105681) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}}
Mar 26 01:38:25.136599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, resource={'id': '14a0aa42-40aa-4508-9af3-cffdffef9818', 'name': '', 'network_id': 'db438906-c74f-4dc8-883c-6e56962e551b', 'project_id': '', 'mac_address': 'fa:16:3e:4e:d4:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e4efd798-3c25-4ea9-bd45-fa7a05aeaef9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '23682989-62bf-4e07-a74f-0cb01077d5ac', 'ip_address': '172.24.5.142'}, {'subnet_id': '1c3e4583-6e63-4953-8050-3341662a5ec6', 'ip_address': '2001:db8::16a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000160849', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-03-26T01:02:52Z', 'updated_at': '2026-03-26T01:02:55Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.136947 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '23682989-62bf-4e07-a74f-0cb01077d5ac 1c3e4583-6e63-4953-8050-3341662a5ec6', 'neutron:network_name': 'neutron-db438906-c74f-4dc8-883c-6e56962e551b', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::16a/64', '172.24.5.142/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.137504 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=14a0aa42-40aa-4508-9af3-cffdffef9818, lrouter_port=lrp-14a0aa42-40aa-4508-9af3-cffdffef9818, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.137899 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.138858 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Traceback (most recent call last):
Mar 26 01:38:25.138858 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.138858 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:25.138858 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.138858 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.138858 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:25.138858 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.138858 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Failed to fix resource 14a0aa42-40aa-4508-9af3-cffdffef9818 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 409, in check_for_inconsistencies
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 318, in _fix_create_update
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1946, in update_router_port
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:25.139949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.140497 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.141102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Mar 26 01:38:25.141102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.141102 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Mar 26 01:38:25.141102 np0000160849 neutron-ovn-maintenance-worker[105681]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4f1b8970-6a08-4da9-bc19-3d1e33f4a316 None None] Maintenance task: Synchronization completed (took 0.32 seconds)
Mar 26 01:38:25.414963 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:25.416641 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:25.457262 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.457654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.458576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:25.458576 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.458576 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:25.458576 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.458576 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.458576 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:25.458576 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.458576 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.459757 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.460382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:25.600547 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:25.601761 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.602080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.602325 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.603193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:25.603193 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.603193 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:25.603193 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.603193 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.603193 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:25.603193 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.603193 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.604882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.605663 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:25.612630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:25.613185 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:25.613774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.614120 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.614829 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:25.614829 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.614829 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:25.614829 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.614829 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.614829 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:25.614829 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.614829 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:25.615720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.616377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:25.616377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.616377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:25.616377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.616377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.616377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:25.616377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.616377 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:25.616729 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.617410 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.618222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:25.618222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.618222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:25.659138 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:25.659882 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:25.660507 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.660861 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.661685 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:25.661685 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.661685 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:25.661685 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.661685 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.661685 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:25.661685 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.661685 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:25.663625 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.664321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:25.666184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.666906 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:25.667699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.667699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:25.702766 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:25.703602 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:25.704302 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.704650 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:25.704761 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.705562 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:25.705562 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.705562 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:25.705562 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.705562 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.705562 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:25.705562 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.705562 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.707339 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:25.708272 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:30.415580 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:30.417090 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:30.418539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=434 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:38:30.454960 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:30.455383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.456228 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:30.456228 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.456228 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:30.456228 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.456228 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.456228 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:30.456228 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.456228 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.457477 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.457922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:30.601177 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:30.602542 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:30.602950 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:30.603132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.604103 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:30.604103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.604103 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:30.604103 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.604103 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.604103 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:30.604103 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.604103 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.605859 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.606929 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:30.613136 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:30.613721 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:30.614338 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:30.614604 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.615347 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:30.615347 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.615347 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:30.615347 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.615347 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.615347 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:30.615347 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.615347 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:30.616680 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:30.617549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:30.617549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.617549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:30.617549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.617549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.617549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:30.617549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.617549 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:30.618440 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.619214 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.620024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:30.620024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.620024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:30.659719 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:30.660255 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:30.661126 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:30.661642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.662598 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:30.662598 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.662598 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:30.662598 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.662598 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.662598 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:30.662598 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.662598 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:30.664163 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.665153 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:30.666502 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.667345 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:30.668134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.668134 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:30.703188 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:30.703861 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:30.704755 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:30.705172 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:30.705397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.706256 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:30.706256 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.706256 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:30.706256 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.706256 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.706256 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:30.706256 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.706256 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.707860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:30.708657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:35.416553 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:35.418220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:35.457753 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:35.457995 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.458585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:35.458585 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.458585 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:35.458585 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.458585 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.458585 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:35.458585 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.458585 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.459683 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.460672 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:35.601914 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:35.603016 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:35.603302 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:35.603540 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.604651 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:35.604651 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.604651 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:35.604651 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.604651 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.604651 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:35.604651 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.604651 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.606276 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.607140 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:35.613595 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:35.614080 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:35.614593 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:35.614879 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.615639 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:35.615639 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.615639 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:35.615639 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.615639 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.615639 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:35.615639 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.615639 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:35.616655 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:35.617200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:35.617200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.617200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:35.617200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.617200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.617200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:35.617200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.617200 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:35.617904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.618399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.618940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:35.618940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.618940 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:35.660103 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:35.660564 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:35.660998 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:35.661237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.661911 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:35.661911 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.661911 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:35.661911 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.661911 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.661911 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:35.661911 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.661911 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:35.663189 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.663919 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:35.664833 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.665646 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:35.666531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.666531 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:35.703781 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:35.704502 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:35.705008 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:35.705393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:35.705620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.706455 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:35.706455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.706455 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:35.706455 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.706455 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.706455 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:35.706455 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.706455 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.708184 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:35.709067 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:40.417195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:40.418704 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:40.452176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:40.452442 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.453281 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:40.453281 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.453281 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:40.453281 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.453281 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.453281 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:40.453281 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.453281 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.454181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.455054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:40.602667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:40.603966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:40.604457 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:40.604624 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.605420 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:40.605420 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.605420 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:40.605420 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.605420 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.605420 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:40.605420 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.605420 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.607132 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.608017 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:40.614086 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:40.614585 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:40.615120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:40.615391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.616172 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:40.616172 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.616172 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:40.616172 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.616172 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.616172 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:40.616172 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.616172 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:40.617468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:40.618250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:40.618250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.618250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:40.618250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.618250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.618250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:40.618250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.618250 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:40.619028 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.619457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.620171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:40.620171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.620171 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:40.660609 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:40.661130 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:40.661787 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:40.662292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.663146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:40.663146 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.663146 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:40.663146 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.663146 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.663146 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:40.663146 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.663146 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:40.664397 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.665085 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:40.665944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.666707 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:40.667573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.667573 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:40.704211 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:40.704667 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:40.705234 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:40.705616 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:40.705916 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.706763 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:40.706763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.706763 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:40.706763 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.706763 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.706763 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:40.706763 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.706763 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.708610 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:40.709513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:45.417852 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:45.419467 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:45.454780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:45.455372 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.456294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:45.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:45.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.456294 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:45.456294 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.456294 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.457457 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.458443 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:45.603383 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:45.604646 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:45.604961 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:45.605211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.606079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:45.606079 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.606079 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:45.606079 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.606079 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.606079 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:45.606079 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.606079 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.607768 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.608142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:45.614537 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:45.614777 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:45.615035 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:45.615186 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.615555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:45.615555 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.615555 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:45.615555 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.615555 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.615555 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:45.615555 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.615555 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:45.616218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:45.616597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:45.616597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.616597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:45.616597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.616597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.616597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:45.616597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.616597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:45.616957 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.617307 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.617657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:45.617657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.617657 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:45.661209 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:45.661519 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:45.662286 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:45.662585 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.663218 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:45.663218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.663218 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:45.663218 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.663218 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.663218 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:45.663218 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.663218 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:45.663831 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.664227 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:45.664599 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.665060 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:45.665552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.665552 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:45.704720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:45.705567 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:45.705776 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:45.706094 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:45.706262 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.706913 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:45.706913 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.706913 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:45.706913 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.706913 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.706913 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:45.706913 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.706913 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.708301 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:45.708782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:50.418514 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:50.420030 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:50.450363 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:50.450570 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.450944 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:50.450944 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.450944 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:50.450944 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.450944 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.450944 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:50.450944 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.450944 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.451762 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.452654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:50.604122 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:50.605470 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:50.605698 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:50.605903 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.606791 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:50.606791 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.606791 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:50.606791 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.606791 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.606791 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:50.606791 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.606791 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.608362 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.608772 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:50.615132 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:50.615661 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:50.616214 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:50.616490 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.617196 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:50.617196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.617196 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:50.617196 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.617196 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.617196 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:50.617196 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.617196 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:50.618426 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:50.619308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:50.619308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.619308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:50.619308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.619308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.619308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:50.619308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.619308 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:50.619959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.620629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.621357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:50.621357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.621357 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:50.661668 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:50.662110 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:50.662703 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:50.663155 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.664004 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:50.664004 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.664004 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:50.664004 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.664004 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.664004 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:50.664004 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.664004 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:50.665394 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.666375 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:50.667070 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.667830 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:50.668698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.668698 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:50.705239 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:50.706408 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:50.707352 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:50.707742 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:50.708115 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.708897 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:50.708897 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.708897 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:50.708897 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.708897 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.708897 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:50.708897 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.708897 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.710005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:50.710529 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:55.419210 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:55.420490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:55.457904 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:55.458111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.458575 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:38:55.458575 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.458575 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:55.458575 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.458575 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.458575 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:55.458575 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.458575 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.459491 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.459956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:55.604572 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:55.605926 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:55.606131 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:55.606287 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.606949 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:55.606949 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.606949 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:55.606949 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.606949 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.606949 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:55.606949 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.606949 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.608926 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.610066 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:55.615796 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:55.616176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:38:55.616601 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:55.616812 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.617282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:38:55.617282 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.617282 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:55.617282 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.617282 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.617282 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:55.617282 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.617282 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:55.617863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:55.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:55.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:55.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:55.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.618367 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:38:55.618620 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.619080 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.619609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:55.619609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.619609 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:55.662260 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:55.662749 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:55.663644 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:55.663927 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.664590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:38:55.664590 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.664590 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:55.664590 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.664590 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.664590 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:55.664590 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.664590 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:38:55.665894 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.666441 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:38:55.667727 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.668802 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:55.669959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.669959 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:38:55.706197 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:38:55.706968 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:38:55.707908 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:55.708344 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:38:55.708591 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.709725 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:38:55.709725 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.709725 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:38:55.709725 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.709725 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.709725 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:38:55.709725 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.709725 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.711283 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:38:55.711817 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:00.420014 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:00.421782 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:00.456114 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:00.456351 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.456753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:00.456753 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.456753 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:00.456753 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.456753 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.456753 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:00.456753 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.456753 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.457626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.458109 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:00.605302 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:00.606438 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:00.606780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:00.607289 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.608202 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:00.608202 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.608202 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:00.608202 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.608202 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.608202 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:00.608202 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.608202 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.609844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.610388 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:00.616259 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:00.616603 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:00.616951 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:00.617197 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.617701 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:00.617701 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.617701 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:00.617701 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.617701 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.617701 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:00.617701 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.617701 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:00.618956 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:00.619460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:00.619460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.619460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:00.619460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.619460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.619460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:00.619460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.619460 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:00.620708 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.621188 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.621678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:00.621678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.621678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:00.662778 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:00.663216 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:00.663737 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:00.664238 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.665005 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:00.665005 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.665005 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:00.665005 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.665005 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.665005 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:00.665005 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.665005 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:00.666190 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.666824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:00.667749 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.668520 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:00.669311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.669311 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:00.706856 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:00.707743 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:00.708176 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:00.708664 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:00.708970 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.710065 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:00.710065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.710065 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:00.710065 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.710065 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.710065 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:00.710065 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.710065 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:00.712138 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:05.420907 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:05.422717 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:05.450587 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:05.450753 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.451207 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:05.451207 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.451207 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:05.451207 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.451207 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.451207 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:05.451207 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.451207 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.451917 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.452508 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:05.606275 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:05.607263 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:05.607546 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:05.607764 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.608543 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:05.608543 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.608543 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:05.608543 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.608543 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.608543 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:05.608543 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.608543 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.610180 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.611013 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:05.616624 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:05.616836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:05.617151 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:05.617320 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.617741 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:05.617741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.617741 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:05.617741 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.617741 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.617741 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:05.617741 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.617741 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:05.618425 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:05.619002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:05.619002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.619002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:05.619002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.619002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.619002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:05.619002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.619002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:05.619316 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.619784 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.620328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:05.620328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.620328 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:05.663551 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:05.664236 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:05.664836 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:05.665280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.666142 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:05.666142 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.666142 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:05.666142 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.666142 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.666142 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:05.666142 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.666142 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:05.667419 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.668112 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:05.669128 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.669888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:05.671052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.671052 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:05.707431 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:05.707955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:05.708550 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:05.708964 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:05.709237 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.709998 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:05.709998 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.709998 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:05.709998 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.709998 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.709998 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:05.709998 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.709998 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.711622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:05.712436 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:10.421613 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:10.423620 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:10.460957 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:10.461166 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.461536 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:10.461536 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.461536 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:10.461536 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.461536 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.461536 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:10.461536 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.461536 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.462291 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.463391 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:10.606919 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:10.608316 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:10.608669 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:10.608915 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.609860 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:10.609860 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.609860 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:10.609860 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.609860 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.609860 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:10.609860 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.609860 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.612873 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:10.617240 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:10.617599 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:10.618118 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:10.618414 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.619146 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:10.619146 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.619146 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:10.619146 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.619146 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.619146 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:10.619146 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.619146 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:10.620310 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:10.620786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:10.620786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.620786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:10.620786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.620786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.620786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:10.620786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.620786 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:10.621935 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.622510 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.623059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:10.623059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.623059 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:10.664150 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:10.664754 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:10.665355 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:10.665723 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.666459 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:10.666459 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.666459 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:10.666459 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.666459 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.666459 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:10.666459 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.666459 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:10.667334 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.668003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:10.668398 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.669124 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:10.669902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.669902 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:10.708084 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:10.708585 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:10.708955 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:10.709315 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:10.709600 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.710321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:10.710321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.710321 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:10.710321 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.710321 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.710321 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:10.710321 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.710321 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.711853 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:10.712648 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:15.422379 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:15.424159 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:15.465688 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:15.465954 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.466578 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:15.466578 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.466578 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:15.466578 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.466578 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.466578 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:15.466578 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.466578 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.467464 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.469038 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:15.607476 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:15.608690 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:15.609133 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:15.609563 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.610298 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:15.610298 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.610298 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:15.610298 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.610298 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.610298 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:15.610298 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.610298 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.611294 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.612201 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:15.617656 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:15.617938 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:15.618527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:15.618892 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.619699 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:15.619699 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.619699 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:15.619699 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.619699 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.619699 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:15.619699 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.619699 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:15.620359 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:15.620863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:15.620863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.620863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:15.620863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.620863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.620863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:15.620863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.620863 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:15.621268 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.621702 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.622203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:15.622203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.622203 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:15.664639 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:15.665595 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:15.666424 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:15.666782 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.668270 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:15.668270 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.668270 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:15.668270 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.668270 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.668270 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:15.668270 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.668270 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:15.670058 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.670576 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:15.671990 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.672535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:15.673079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.673079 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:15.708566 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:15.709276 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:15.709826 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:15.710353 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:15.710712 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.711834 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:15.711834 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.711834 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:15.711834 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.711834 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.711834 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:15.711834 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.711834 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.713678 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:15.714711 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:20.423074 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:20.424471 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:20.457939 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:20.458412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.459239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:20.459239 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.459239 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:20.459239 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.459239 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.459239 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:20.459239 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.459239 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.460158 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.461082 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:20.608124 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:20.609393 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:20.609757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:20.610015 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.610932 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:20.610932 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.610932 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:20.610932 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.610932 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.610932 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:20.610932 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.610932 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.611848 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.612752 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:20.618152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:20.618527 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:20.618961 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:20.619174 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.619740 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:20.619740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.619740 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:20.619740 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.619740 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.619740 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:20.619740 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.619740 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:20.620933 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:20.621461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:20.621461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.621461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:20.621461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.621461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.621461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:20.621461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.621461 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:20.622637 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.623111 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.623618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:20.623618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.623618 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:20.665191 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:20.665630 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:20.666220 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:20.666534 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.667314 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:20.667314 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.667314 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:20.667314 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.667314 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.667314 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:20.667314 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.667314 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:20.668468 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.669178 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:20.669586 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.670383 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:20.671211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.671211 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:20.709371 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:20.709778 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:20.710126 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:20.710358 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:20.710507 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.711003 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:20.711003 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.711003 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:20.711003 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.711003 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.711003 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:20.711003 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.711003 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.712154 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:20.712904 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:25.423662 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:25.425247 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:25.457942 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:25.458280 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.458888 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:25.458888 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.458888 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:25.458888 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.458888 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.458888 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:25.458888 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.458888 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.459779 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.460355 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:25.608774 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:25.610006 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:25.610390 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:25.610626 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.611513 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:25.611513 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.611513 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:25.611513 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.611513 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.611513 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:25.611513 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.611513 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.613421 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.614230 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:25.618529 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:25.618780 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:25.619274 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:25.619546 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.620205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:25.620205 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.620205 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:25.620205 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.620205 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.620205 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:25.620205 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.620205 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:25.621167 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:25.621528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:25.621528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.621528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:25.621528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.621528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.621528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:25.621528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.621528 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:25.622282 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.622644 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.623024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:25.623024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.623024 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:25.665720 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:25.666208 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:25.666738 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:25.667114 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.667869 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:25.667869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.667869 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:25.667869 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.667869 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.667869 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:25.667869 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.667869 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:25.669205 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.670081 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:25.671002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.671775 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:25.672642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.672642 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:25.709859 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:25.710398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:25.710773 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:25.711127 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:25.711348 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.712482 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:25.712482 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.712482 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:25.712482 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.712482 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.712482 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:25.712482 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.712482 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.714061 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:25.714925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:30.424534 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:30.425615 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:30.427160 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG dbcounter [-] [105681] Writing DB stats neutron:SELECT=240 {{(pid=105681) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Mar 26 01:39:30.457645 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:30.458175 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.459043 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:30.459043 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.459043 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:30.459043 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.459043 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.459043 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:30.459043 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.459043 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.459900 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.460638 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:30.609466 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:30.610677 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:30.611113 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:30.611113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.611844 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:30.611844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.611844 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:30.611844 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.611844 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.611844 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:30.611844 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.611844 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.613640 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.614509 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:30.618931 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:30.619492 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:30.620049 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:30.620343 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.621113 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:30.621113 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.621113 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:30.621113 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.621113 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.621113 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:30.621113 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.621113 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:30.622720 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.623922 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:30.624636 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.625472 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:30.666167 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:30.666805 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:30.667368 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:30.667718 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.668512 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:30.668512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.668512 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:30.668512 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.668512 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.668512 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:30.668512 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.668512 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:30.669878 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.670553 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:30.671674 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.672399 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:30.673387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.673387 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:30.710398 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:30.711195 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:30.712359 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:30.712719 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:30.713219 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.714547 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:30.714547 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.714547 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:30.714547 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.714547 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.714547 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:30.714547 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.714547 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.716696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:30.718323 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:35.425331 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:35.427685 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:35.463354 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:35.463632 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:35.464179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.464179 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:35.464179 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.464179 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.464179 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:35.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.464179 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.465296 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.466234 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:35.610228 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:35.611625 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:35.612000 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:35.612259 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.613292 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:35.613292 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.613292 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:35.613292 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.613292 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.613292 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:35.613292 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.613292 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.615020 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.615846 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:35.620078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:35.620078 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:35.620496 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:35.620803 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.621687 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:35.621687 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.621687 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:35.621687 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.621687 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.621687 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:35.621687 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.621687 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:35.622447 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.624012 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:35.624871 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.625824 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:35.666892 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:35.667292 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:35.667768 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:35.668224 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.669002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:35.669002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.669002 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:35.669002 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.669002 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.669002 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:35.669002 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.669002 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:35.669798 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.670544 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:35.670961 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.671790 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:35.672643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.672643 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:35.711015 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:35.711539 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:35.712038 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:35.712405 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:35.712629 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.713622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:35.713622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.713622 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:35.713622 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.713622 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.713622 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:35.713622 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.713622 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.715882 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:35.716688 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:40.426422 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:40.430219 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:40.464373 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:40.464789 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:40.466505 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:40.466988 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.467378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:40.467378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.467378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.467378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:40.467378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.467378 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:40.611163 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:40.612429 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:40.612757 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:40.613002 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.613800 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:40.613800 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.613800 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:40.613800 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.613800 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.613800 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:40.613800 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.613800 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.615456 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.616758 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:40.620611 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:40.620908 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:40.621152 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:40.621321 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.621696 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:40.621696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.621696 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:40.621696 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.621696 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.621696 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:40.621696 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.621696 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:40.622239 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.623117 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:40.623925 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.624898 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:40.667189 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:40.667792 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:40.668248 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:40.668470 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.669193 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:40.669193 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.669193 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:40.669193 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.669193 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.669193 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:40.669193 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.669193 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:40.670593 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.671273 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:40.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.672951 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:40.673730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.673730 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:40.711493 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:40.711789 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:40.712108 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:40.712309 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:40.712412 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.712886 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:40.712886 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.712886 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:40.712886 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.712886 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.712886 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:40.712886 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.712886 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.714054 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:40.714841 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:45.427840 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:45.429848 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:45.467752 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:45.468162 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.469332 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Traceback (most recent call last):
Mar 26 01:39:45.469332 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.469332 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:45.469332 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.469332 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.469332 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:45.469332 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.469332 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-a5c26569-6bf9-4687-a757-9a07ea500b51 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 778, in check_fdb_aging_settings
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.470380 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.471032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:45.611810 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:45.613318 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-623e19f7-2e8b-459b-a559-a52aeb1143fe, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:45.613696 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e4efd798-3c25-4ea9-bd45-fa7a05aeaef9, routes=[], if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:45.614327 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.615568 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:45.615568 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.615568 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:45.615568 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.615568 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.615568 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:45.615568 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.615568 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 870, in check_router_default_route_empty_dst_ip
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.617622 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.618524 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:45.620909 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:45.621328 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=105681) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1070}}
Mar 26 01:39:45.621812 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:45.622032 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.622597 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Mar 26 01:39:45.622597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.622597 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:45.622597 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.622597 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.622597 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:45.622597 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.622597 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:45.623329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:45.624047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:45.624047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.624047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:45.624047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.624047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.624047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:45.624047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.624047 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1074, in set_fip_distributed_flag
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Mar 26 01:39:45.624329 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                                                           ^^^^^^^^
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.624822 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.625382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:45.625382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.625382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:45.667828 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:45.668336 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:45.669120 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:45.669555 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.670535 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Traceback (most recent call last):
Mar 26 01:39:45.670535 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.670535 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:45.670535 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.670535 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.670535 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:45.670535 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.670535 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Mar 26 01:39:45.671411 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.672222 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.command 
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-00864ff4-5728-4f2a-a9dc-705374a6c272 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1129, in update_ha_failover
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Mar 26 01:39:45.672654 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.673465 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:45.674382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.674382 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics 
Mar 26 01:39:45.711966 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105681) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:638}}
Mar 26 01:39:45.712866 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105681) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Mar 26 01:39:45.713118 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:45.713490 np0000160849 neutron-ovn-maintenance-worker[105681]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105681) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Mar 26 01:39:45.713590 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.714267 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Traceback (most recent call last):
Mar 26 01:39:45.714267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.714267 np0000160849 neutron-ovn-maintenance-worker[105681]:     txn.results.put(txn.do_commit())
Mar 26 01:39:45.714267 np0000160849 neutron-ovn-maintenance-worker[105681]:                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.714267 np0000160849 neutron-ovn-maintenance-worker[105681]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.714267 np0000160849 neutron-ovn-maintenance-worker[105681]:     raise RuntimeError(msg)
Mar 26 01:39:45.714267 np0000160849 neutron-ovn-maintenance-worker[105681]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.714267 np0000160849 neutron-ovn-maintenance-worker[105681]: 
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics [None req-342a7bcd-c15c-4e87-bfda-3ec6b6dcb5d8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics Traceback (most recent call last):
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 290, in run
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     work()
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 64, in __call__
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 178, in decorator
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(*args, **kwargs)
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 792, in update_mac_aging_settings
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.715389 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     next(self.gen)
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     with self.create_transaction(
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     self.result = self.commit()
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise result.ex
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics     raise RuntimeError(msg)
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Mar 26 01:39:45.716181 np0000160849 neutron-ovn-maintenance-worker[105681]: ERROR futurist.periodics